Sorry for the delay getting back to you Dawid. That problem is actually because of the Asciidoctor version. The jekyll-asciidoc plugin will install Asciidoctor if it is not already installed, and it installs a version where the way the links are constructed is different and breaks our validation.
More details about why this happens (if you’re curious) is in https://issues.apache.org/jira/browse/SOLR-12786?focusedCommentId=16622115&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16622115 Both the Jenkins script used for Jenkins Ref Guide jobs (./dev-tools/scripts/jenkins.build.ref.guide.sh) and the Ref Guide README (./solr/solr-ref-guide/README.adoc) show examples of how to make sure the right Asciidoctor version is installed - the easiest is to install the Asciidoctor gem version we want first. Let me see if I can insert a line to install before the jekyll-asciidoc gem and see if that fixes it. Cassandra On Oct 10, 2019, 2:22 AM -0500, Dawid Weiss <dawid.we...@gmail.com>, wrote: > Ping, ping. I wanted to finalize the build of solr ref guide since I > started it. Almost everything is working on the branch but I can't get > minor differences to work and I believe they're due to a different > jekyll version (than that mentioned in the docs). > > Specifically, the invalid links are because of asciidoc sections like > this (in the processed resource-and-plugin-loading.adoc): > > === solr_home/lib > > In bare bones html (pure asciidoctor) this gets emitted as: > > <h3 id="solr_home-lib">solr_home/lib</h3> > > but when compiled via jekyll this becomes: > > <h3 id="solr_homelib">solr_home/lib</h3> > > which I can't really explain. > > Cassandra what's the exact version of jekyll that runs the compilation > that is working for you? > > D. > > On Fri, Oct 4, 2019 at 11:42 AM Dawid Weiss <dawid.we...@gmail.com> wrote: > > > > Seems like pygments is to blame for the python requirement... I didn't > > check but there seem to be ruby-only > > highlighters for jekyll as well: > > > > https://jekyll-windows.juthilo.com/3-syntax-highlighting/ > > > > On Fri, Oct 4, 2019 at 11:39 AM Dawid Weiss <dawid.we...@gmail.com> wrote: > > > > > > Hi Cassandra, > > > > > > Apologies this took so long -- I wasn't familiar with these > > > site-generation tools and the whole ecosystem is rather... fragile :) > > > After a few attempts at using gradle plugins I eventually leaned > > > towards using asciidoctor and jekyll explicitly (so that we know which > > > versions are being used and don't have to rely on dependencies). > > > > > > I got bare bone html checking working, PDF generation working and site > > > generation working although the final link check currently fail for me > > > with a bunch of errors. This works for me on Windows... on Linux I get > > > site-generation generate a strange error from within jekyll: > > > > > > Conversion error: Jekyll::AsciiDoc::Converter encountered an error > > > while converting 'about-filters.adoc': > > > Bad file descriptor - /usr/bin/python2 > > > > > > I could install python but I don't see why it'd need it. Perhaps there > > > is something in the docs that would avoid using python altogether but > > > I haven't had the time to look into it. > > > > > > Please feel free to check out the jira/SOLR-13452_gradle_7_refguide > > > branch and try to run: > > > > > > ./gradlew -p solr/solr-ref-guide buildPdf buildSite > > > > > > There is a lot of room for improvement -- from property substitution, > > > through how the "tools" are handled at the moment to task naming but I > > > left this for the future. The initial step would be probably to get > > > the site generation running on Linux/ Macs but I'd gladly hand it over > > > back to you -- I can help with Gradle but a the rest of those tools > > > are a mistery to me. > > > > > > Dawid > > > > > > On Fri, Sep 27, 2019 at 7:53 PM Dawid Weiss <dawid.we...@gmail.com> wrote: > > > > > > > > > > > > No problem. I will get it to work entirely, but not before next week - > > > > I am away for the weekend. > > > > > > > > Dawid > > > > > > > > On Fri, Sep 27, 2019, 16:17 Cassandra Targett <casstarg...@gmail.com> > > > > wrote: > > > > > > > > > > Thanks Dawid for working on this! I’ve been a bit swamped the last > > > > > couple of days but will take a look today at what you’ve been able to > > > > > do so far and see where we might need to go from here. > > > > > > > > > > Cassandra > > > > > On Sep 26, 2019, 7:25 AM -0500, Dawid Weiss <dawid.we...@gmail.com>, > > > > > wrote: > > > > > > > > > > I agree. Although I also understand the concern of trying to merge the > > > > > changes while we're in the transition period... it'd be hell. I'd say > > > > > move as much stuff as possible with the current folder structure (and > > > > > ignore what cannot be ported easily) then switch as soon as possible > > > > > to gradle and hack the old cruft with a chainsaw... > > > > > > > > > > D. > > > > > > > > > > On Thu, Sep 26, 2019 at 2:13 PM Erick Erickson > > > > > <erickerick...@gmail.com> wrote: > > > > > > > > > > > > > > > Of course I’ll completely defer to Dawid and Mark (well and anybody > > > > > else actually, you know, doing _work_), but just can’t resist chiming > > > > > in ;). > > > > > > > > > > My vote would be to “do it the Gradle way”. Yes, it’s a PITA to learn > > > > > new stuff and I won’t like it. Tough. I see no reason to carry a > > > > > bunch of cruft around because “that the way we always did it”. > > > > > > > > > > If we lose functionality, that’s a different discussion, starting > > > > > with “do we need that functionality". But jumping through hoops and > > > > > having to maintain that awkwardness forever going forward just > > > > > because we forced the Ant structure on Gradle strikes me as a poor > > > > > trade off. > > > > > > > > > > That said, I’m not doing the work so I really have no vote. But don’t > > > > > strain to do it the old way on my account ;) > > > > > > > > > > Erick > > > > > > > > > > P.S. Thanks Dawid for jumping in! > > > > > > > > > > On Sep 26, 2019, at 3:57 AM, Dawid Weiss <dawid.we...@gmail.com> > > > > > wrote: > > > > > > > > > > I pushed it in to Lucene repo (it's on Cassandra's refguide branch > > > > > anyway, so shouldn't interfere with anything else); seems like it's in > > > > > better shape than previous code anyway (those questions I asked about > > > > > the nature of the gradle port still hold though). > > > > > > > > > > I got as far as building initial bare-bones HTML. > > > > > > > > > > .\gradlew -p solr\solr-ref-guide clean bareBonesHtmlValidation > > > > > > > > > > I don't know anything about the pipeline involved (asciidoctor, etc.) > > > > > so it's very likely some attributes will have to be corrected later > > > > > on. > > > > > > > > > > Dawid > > > > > > > > > > On Wed, Sep 25, 2019 at 9:14 PM Dawid Weiss <dawid.we...@gmail.com> > > > > > wrote: > > > > > > > > > > > > > > > I looked at the solr ref guide build and started converting it to > > > > > Gradle but have a question to Mark (because he coordinates the > > > > > effort). > > > > > > > > > > What immediately jumps into face is the decision problem -- do we want > > > > > to emulate what ant does at the moment or do we want to clean it up > > > > > (breaking file/ folder structure and causing incompatibility with ant > > > > > build). > > > > > > > > > > I went the "compatible" way and started porting ant tasks but it's > > > > > quite awkward. For example -- there are template properties that refer > > > > > to ivy version properties... we could emulate/ compute these but it's > > > > > a pain. The way the module is currently structured is also awkward - > > > > > it'd be more natural to have a separate java project with the "tools" > > > > > required to compile extra stuff and just reference it from the manual > > > > > build (and this would be a plain module, not a java module). This > > > > > would limit the need for customizing source sets, classpaths, etc. > > > > > > > > > > My few initial tasks syncing sources, setting up infrastructure to > > > > > filter templates and compiling the required tools are here: > > > > > https://github.com/apache/lucene-solr/compare/jira/SOLR-13452_gradle_7_refguide...dweiss:jira/SOLR-13452_gradle_7_refguide?expand=1 > > > > > > > > > > I'll stop and wait for feedback (especially on the ivy versions issue) > > > > > before I resume. > > > > > > > > > > Dawid > > > > > > > > > > On Wed, Sep 25, 2019 at 6:20 PM Dawid Weiss <dawid.we...@gmail.com> > > > > > wrote: > > > > > > > > > > > > > > > Never mind, I've got it. > > > > > > > > > > D. > > > > > > > > > > On Wed, Sep 25, 2019 at 7:59 AM Dawid Weiss <dawid.we...@gmail.com> > > > > > wrote: > > > > > > > > > > > > > > > Hi Cassandra, > > > > > > > > > > I’m more than happy to share more details our current build so we can > > > > > replicate some of the above steps, but I’m stuck without a lot more > > > > > basic Gradle skills that I don’t have time to acquire with > > > > > day-job/personal life commitments. I put it into a separate branch so > > > > > we could iterate a little easier, can anyone help? > > > > > > > > > > > > > > > Where is this branch you made changes on? If you can point me at the > > > > > corresponding ant code I'll try to help you out. > > > > > > > > > > Dawid > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org > > > > > For additional commands, e-mail: dev-h...@lucene.apache.org > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org > > > > > For additional commands, e-mail: dev-h...@lucene.apache.org > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org > > > > > For additional commands, e-mail: dev-h...@lucene.apache.org > > > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org > For additional commands, e-mail: dev-h...@lucene.apache.org >
