I concur with all the preceding discussion, about removing the
zipped-files feature, and about the desirability of asynchronous loading
if that is practical.
Separate RFE: the UI should indicate if/while the results are incomplete.
There's a TODO in AbstractIndexWriter.java
472 if (!searchIndex.isEmpty()) { // TODO: write to disk straight
I'm pleased to see this line go:
318 tree.add(new RawHtml("<!--[if IE]>"));
We need a (separate?) plan of action for documenting this change and
recommending that webservers delivering big API index files should be
configured to use compression.
I intend to let this RFR sit here for at least 2 weeks.
The perceived severity of this change should not be underestimated.
I think it is equally important to get this pushed early in the release,
so that it can
be tested with real-world docs, such as the EA docs for JDK 15.
-- Jon
On 01/28/2020 07:55 AM, Pavel Rappo wrote:
Hello,
Please review the change for https://bugs.openjdk.java.net/browse/JDK-8237909:
http://cr.openjdk.java.net/~prappo/8237909/webrev.00/
This change removes the "zipped index files" feature, which was introduced as
part of 8141492: Implement search feature in javadoc.
The "zipped index files" feature consists of generating the zipped index files
on the back end, and fetching & unzipping mechanics on the front end.
When documenting source files, the standard doclet accumulates index which is
later used by the JavaScript code serving the interactive search. The index
is written in two formats, .js (JavaScript) and .json (JSON). The latter is
then zipped.
When a browser accesses the pages using "http://"; urls, the .zip index files are
transferred using XHR. Those files are then unzipped by the browser, using the
JSZip library, and parsed as JSON. If the transfer of the .zip index files fails
for whatever reason, the browser falls back on the alternative mechanism. This
mechanism transfers the .js index files by referring to them from dynamically
inserted <script src="... .js"> elements. Those files then are not additionally
parsed, as they are already data hardcoded in JavaScript code.
One of the reasons the .zip index files transfer may fail is using javadoc pages
in the "standalone" mode. When a browser accesses "file://" urls, there's no
HTTP server to send the XHR requests to. So the fallback mechanism kicks in and
the browser loads the .js index files instead.
Analysis
========
From what I understand, the original intent was to reduce the transfer size of
the index files. The observations made during the recent upgrade of JSZip
(JDK-8236700) suggest that the feature is not working as intended. It is not
clear if it ever did. The proposal is to remove it for the following reasons:
1. The feature in its current state does more harm than good (see JDK-8236922)
2. Fixing, debugging, testing, and evolving require expertise beyond that of
typical for the javadoc area
3. The feature significantly complicates the front end and less so the back end
code
4. The feature relies on the 3rd party libraries, which require tracking &
periodical upgrades
5. The difference in size between the .zip and .js files is not that big (see
below)
6. The index files are transferred once and then used from cache
7. Modern HTTP servers provide compression. This makes the net result
virtually the same, compare:
| (current) js + zip, MB | (proposal) js files, MB
------------------+------------------------+------------------------
no compression 7.4 5.8
HTTP compression 2.7 1.4
Had this feature worked as intended, we would always transfer only the zipped
index files and the transfer size would not depend on whether the server uses
HTTP compression. But does this really outweigh the reasons stated above?
Summing all up. Removing the zipped index files feature will make the overall
interactive search feature (JDK-8141492) more robust. It will be less
complicated, have fewer dependencies (JSZip, JSZip Utils), and will push the
optimization down to HTTP.
Testing
=======
Here is how I tested this change.
1. make clean && make docs
2. Standalone test
2.1. Opened the browser at file://...images/docs/index.html
3. HTTP test
3.1. Started an HTTP server at build/...images/docs
3.2. Opened the browser at http://localhost...images/docs/index.html
Browser cache was cleared each time immediately before accessing the index.html
page.
In both cases I checked that no zipped index files or the related JavaScript
libraries were accessed, and that the search worked as intended.
I also tried to access the resulting javadoc pages, served by an HTTP server on
my laptop, from a couple of mobile devices, all of which were on the same WiFi
network. Everything worked as intended.
Thanks,
-Pavel
