Bug#1064824: node-d3: fails to export map and other functions

2024-03-05 Thread Julian Gilbey 
On Tue, Mar 05, 2024 at 05:39:00PM +0530, Nilesh Patra wrote:
> On Mon, Mar 04, 2024 at 09:18:01PM +, Julian Gilbey wrote:
> [...]
> > (!) Conflicting re-exports
> > "index.js" re-exports "map" from both 
> > "../../../usr/share/nodejs/d3-array/src/index.js" and 
> > "../../../usr/share/nodejs/d3-collection/src/index.js" (will be ignored).
> > created dist/d3.min.js in 4.2s
> > -
> 
> I have pushed a commit to salsa that hopefully fixes this - can you please 
> try with the same and see if that
> helps you somewhat?

That's perfect, thanks!  And such a simple solution :-) (Though I
don't pretend to understand how the patch works!)  The taskflow
webserver now works exactly as intended (as far as I can tell).

> > So it's specifically "map" that is problematic, and I just happen to
> > have stumbled upon it: d3 v5 depends on d3-array version 1, but the
> > version of node-d3-array in unstable is 3.2.0+~cs5.0.6-2, and this is
> > causing the conflict.
> > 
> > I don't know the best way to fix this.  node-d3-array version was
> > upgraded from 1.2.4 to 3.x about two years ago, so d3 would have had
> > this bug since then, but I'm the first one to stumble upon it :-/
> >
> > Perhaps we could package node-d3-array-1 (version 1.2.4) and have
> > node-d3 build-depends on that?
> 
> I tried to embed it and realised it is creating an unholy mess. I got it 
> working eventually
> but it can open a can of worms sometime later. Maybe packaging the older 
> version
> would be the way to go if my fix above does not work.
> 
> I've added yadd to the thread for more qualified advice.

It might be a good idea to do this anyway, but given that no-one else
has noticed a problem in > 2 years suggests it's not worth the effort.

Best wishes,

   Julian

Bug#1064824: node-d3: fails to export map and other functions

2024-03-05 Thread Nilesh Patra 
On Mon, Mar 04, 2024 at 09:18:01PM +, Julian Gilbey wrote:
> index.js → dist/d3.js...
> (!) Conflicting re-exports
> "index.js" re-exports "map" from both 
> "../../../usr/share/nodejs/d3-array/src/index.js" and 
> "../../../usr/share/nodejs/d3-collection/src/index.js" (will be ignored).
> created dist/d3.js in 1.9s
> 
> index.js → dist/d3.min.js...
> (!) Conflicting re-exports
> "index.js" re-exports "map" from both 
> "../../../usr/share/nodejs/d3-array/src/index.js" and 
> "../../../usr/share/nodejs/d3-collection/src/index.js" (will be ignored).
> created dist/d3.min.js in 4.2s
> -

I have pushed a commit to salsa that hopefully fixes this - can you please try 
with the same and see if that
helps you somewhat?

> So it's specifically "map" that is problematic, and I just happen to
> have stumbled upon it: d3 v5 depends on d3-array version 1, but the
> version of node-d3-array in unstable is 3.2.0+~cs5.0.6-2, and this is
> causing the conflict.
> 
> I don't know the best way to fix this.  node-d3-array version was
> upgraded from 1.2.4 to 3.x about two years ago, so d3 would have had
> this bug since then, but I'm the first one to stumble upon it :-/
>
> Perhaps we could package node-d3-array-1 (version 1.2.4) and have
> node-d3 build-depends on that?

I tried to embed it and realised it is creating an unholy mess. I got it 
working eventually
but it can open a can of worms sometime later. Maybe packaging the older version
would be the way to go if my fix above does not work.

I've added yadd to the thread for more qualified advice.

Best,
Nilesh


signature.asc
Description: PGP signature

Bug#1064824: node-d3: fails to export map and other functions

2024-03-04 Thread Julian Gilbey 
On Mon, Mar 04, 2024 at 09:18:01PM +, Julian Gilbey wrote:
> [...]
> So it's specifically "map" that is problematic, and I just happen to
> have stumbled upon it: d3 v5 depends on d3-array version 1, but the
> version of node-d3-array in unstable is 3.2.0+~cs5.0.6-2, and this is
> causing the conflict.
> 
> I don't know the best way to fix this.  node-d3-array version was
> upgraded from 1.2.4 to 3.x about two years ago, so d3 would have had
> this bug since then, but I'm the first one to stumble upon it :-/
> 
> Perhaps we could package node-d3-array-1 (version 1.2.4) and have
> node-d3 build-depends on that?

I was just wondering if there's a simpler way than having a new
package: have d3-array 1.2.4 being an unexported component of the
node-d3 source package, and instead of Build-Depends: node-d3-array,
have a Build-Conflicts: node-d3-array.  But then I discovered that the
binary package node-d3 depends on node-d3-array, as do 7 other
packages in testing (I haven't checked unstable).  I'm guessing that
most of those dependencies probably need version 1.x of d3-array, so
it may be that having a node-d3-array-1 (or -v1) package is actually
the way forward in this situation.

Best wishes,

   Julian

Bug#1064824: node-d3: fails to export map and other functions

2024-03-04 Thread Julian Gilbey 
Hi Nilesh,

On Tue, Mar 05, 2024 at 02:06:08AM +0530, Nilesh Patra wrote:
> > This gives lots of differences still; stripping down to just the
> > differences still has many, many differences: some new exports not in
> > the original d3, and some lost exports; the list begins:
> > $ diff -u /tmp/d3-npm.exports.trimmed /tmp/d3-debian.exports.trimmed
> >
> > +exports.Adder = Adder;
> > -exports.bisect = bisectRight;
> > +exports.bin = bin;
> > +exports.bisect = bisect;
> > +exports.bisectCenter = bisectCenter;
> > +exports.blur2 = blur2;
> > +exports.blur = blur;
> > +exports.blurImage = blurImage;
> > +exports.count = count;
> > -exports.csvFormatRow = csvFormatRow;
> > -exports.csvFormatValue = csvFormatValue;
> 
> $ cat /tmp/d3-debian.exports.trimmed | egrep --color 
> '(bisectRight|csvFormatRow|csvFormatValue)'
> exports.bisectRight = bisectRight;
> exports.csvFormatRows = csvFormatRows;
> 
> Some of the diff entries are false positive -- it is just that entries are 
> not identical across these
> files and despite sorting them, you do not get the exact picture of the diff 
> in exports.

Well spotted, thanks!  I'll snip the discussion of csvFormatValue...

> [...]
> Which is why. Seems the versions of dev dependencies have not been 
> appropriately tightened by the upstream
> so we are running into weird surprises like these. Re-compiling node-d3 again 
> now should fixup this export however.

Great!

> These minor deltas in exports are more or less due to
> version differences in different d3 plugins.

> > ...
> > Background to this: I'm trying to package a new package which provides
> > a web server to visualise some data.  The package includes a few
> > precompiled JavaScript libraries obtained from npmjs.com, and the
> > server works fine with them.  But following Debian policy, I need to
> > replace those with the source packages.  And the server then doesn't
> > work.
> >
> > The JavaScript libraries which the package uses are: d3 v5.16.0,
> > d3-tip, apparently v0.9.1, along with jQuery and bootstrap4.  I have
> > replaced all of these with the versions in the corresponding Debian
> > packages (and I've just uploaded a new version of d3-tip, thinking
> > that that was the cause of the bug).
> >
> > When visiting the served web page, the console log gives the error
> > message:
> >
> > Uncaught (in promise) TypeError: t.map is not a function
> >n http://localhost:8080/js/d3/d3-tip.min.js:1
> >main http://localhost:8080/js/index.js:848
> >async* http://localhost:8080/js/index.js:993
> >
> > (This has changed from the original bug report as the minimised new
> > version of d3-tip has t.map instead of h.map.)
> >
> > d3-tip.js requires d3-collection, from which it calls a map function.
> > I tried replacing d3-tip.min.js with the pre-packaged version rather
> > than the (newly built) Debian version, but that did not help.  I
> > reverted that change and instead replaced d3.v5.min.js (which is a
> > copy of /usr/share/nodejs/d3/dist/d3.min.js) with the version provided
> > by upstream, which is a verbatim copy of the npmjs.com d3.min.js, and
> > the server then worked perfectly.  So this told me that it is the
> > Debian compiled d3 which is not working correctly.
> 
> Julian, I am very confused by that wording - from what I could gauge, your
> target package does not work with debian libs but it does work with npmjs, 
> yes?

I'm sorry I wasn't clear.

Yes, that's it: I'm trying to package taskflow for Debian (it's a
dependency of something else that I'm trying to package), and it
provides a profiler visualisation tool in the form of a webserver.

The upstream package (https://github.com/taskflow/taskflow) has
d3.v5.min.js and d3-tip.min.js included (in tfprof/js/d3), and these
are bitwise identical to the npmjs versions (versions 5.16.0 and
0.9.1).  The webserver works using these versions.  To satisfy Debian
policy, I replaced these with the versions found in libjs-d3-tip and
node-d3, but then the webserver fails to produce a usuable
visualisation.

My exploration, described above, was that "map" was not exported from
d3-collection, and that led me to explore whether this was a unique
missing export or not; I discovered that it was not.

> In that case linking your target package and listing the exact steps to
> that error can help someone debug it in more detail as to what might be 
> missing.

I've just rebuilt node-d3 locally from (Debian) source on an unstable
schroot, and I think I may have found the source of the bug: the build
reads:

-
   dh_auto_build --buildsystem=nodejs
No build command found, searching known files
No build command found, searching known files
Found debian/nodejs/d3-sankey/build
cd ./d3-sankey && sh -ex ../debian/nodejs/d3-sankey/build
+ rollup -c

src/index.js → dist/d3-sankey.js...
created dist/d3-sankey.js in 120ms

src/index.js → dist/d3-sankey.min.js...
created dist/d3-sankey.min.js in 561ms
Found debian/nodejs/./build
cd ./. && sh

Bug#1064824: node-d3: fails to export map and other functions

2024-03-04 Thread Nilesh Patra 
> This gives lots of differences still; stripping down to just the
> differences still has many, many differences: some new exports not in
> the original d3, and some lost exports; the list begins:
> $ diff -u /tmp/d3-npm.exports.trimmed /tmp/d3-debian.exports.trimmed
>
> +exports.Adder = Adder;
> -exports.bisect = bisectRight;
> +exports.bin = bin;
> +exports.bisect = bisect;
> +exports.bisectCenter = bisectCenter;
> +exports.blur2 = blur2;
> +exports.blur = blur;
> +exports.blurImage = blurImage;
> +exports.count = count;
> -exports.csvFormatRow = csvFormatRow;
> -exports.csvFormatValue = csvFormatValue;

$ cat /tmp/d3-debian.exports.trimmed | egrep --color 
'(bisectRight|csvFormatRow|csvFormatValue)'
exports.bisectRight = bisectRight;
exports.csvFormatRows = csvFormatRows;

Some of the diff entries are false positive -- it is just that entries are not 
identical across these
files and despite sorting them, you do not get the exact picture of the diff in 
exports.

However I did not find csvFormatValue and I dug a little bit into it. Seems 
this is sourced from node-d3-dsv
and exported with index.js: https://github.com/d3/d3/blob/v5.16.0/index.js

And checking on current unstable

| [/tmp/node-d3-dsv]$ git checkout debian/1.2.0+_1.2.3-1
| HEAD is now at 76229a9 releasing package node-d3-dsv version 1.2.0+~1.2.3-1
| [76229a9][/tmp/node-d3-dsv]$ grep -irn csvFormatValue
| types-d3-dsv/index.d.ts:195:// csvFormatValue(...) 

| types-d3-dsv/index.d.ts:202:export function csvFormatValue(value: string): 
string;
| src/csv.js:11:export var csvFormatValue = csv.formatValue;
| src/index.js:2:export {csvParse, csvParseRows, csvFormat, csvFormatBody, 
csvFormatRows, csvFormatRow, csvFormatValue} from "./csv.js";

So this should work, weird. I wanted to dig in, as per the build log: 
https://buildd.debian.org/status/fetch.php?pkg=node-d3=all=5.16.0%2B%7Ecs5.28.10-1=1693715544=0

| Selecting previously unselected package node-d3-dsv.
| Preparing to unpack .../317-node-d3-dsv_1.1.1-9_all.deb ...
| Unpacking node-d3-dsv (1.1.1-9) ...
| Selecting previously unselected packa

And

| [76229a9][/tmp/node-d3-dsv]$ git checkout debian/1.1.1-9   
| Previous HEAD position was 76229a9 releasing package node-d3-dsv version 
1.2.0+~1.2.3-1
| HEAD is now at 84fbfce releasing package node-d3-dsv version 1.1.1-9
| [84fbfce][/tmp/node-d3-dsv]$ grep -irn csvFormatValue | wc -l
| 0

Which is why. Seems the versions of dev dependencies have not been 
appropriately tightened by the upstream
so we are running into weird surprises like these. Re-compiling node-d3 again 
now should fixup this export however.

:-(

These minor deltas in exports are more or less due to
version differences in different d3 plugins.

> ...
> Background to this: I'm trying to package a new package which provides
> a web server to visualise some data.  The package includes a few
> precompiled JavaScript libraries obtained from npmjs.com, and the
> server works fine with them.  But following Debian policy, I need to
> replace those with the source packages.  And the server then doesn't
> work.
>
> The JavaScript libraries which the package uses are: d3 v5.16.0,
> d3-tip, apparently v0.9.1, along with jQuery and bootstrap4.  I have
> replaced all of these with the versions in the corresponding Debian
> packages (and I've just uploaded a new version of d3-tip, thinking
> that that was the cause of the bug).
>
> When visiting the served web page, the console log gives the error
> message:
>
> Uncaught (in promise) TypeError: t.map is not a function
>n http://localhost:8080/js/d3/d3-tip.min.js:1
>main http://localhost:8080/js/index.js:848
>async* http://localhost:8080/js/index.js:993
>
> (This has changed from the original bug report as the minimised new
> version of d3-tip has t.map instead of h.map.)
>
> d3-tip.js requires d3-collection, from which it calls a map function.
> I tried replacing d3-tip.min.js with the pre-packaged version rather
> than the (newly built) Debian version, but that did not help.  I
> reverted that change and instead replaced d3.v5.min.js (which is a
> copy of /usr/share/nodejs/d3/dist/d3.min.js) with the version provided
> by upstream, which is a verbatim copy of the npmjs.com d3.min.js, and
> the server then worked perfectly.  So this told me that it is the
> Debian compiled d3 which is not working correctly.

Julian, I am very confused by that wording - from what I could gauge, your
target package does not work with debian libs but it does work with npmjs, yes?

In that case linking your target package and listing the exact steps to
that error can help someone debug it in more detail as to what might be missing.

Best,
Nilesh


signature.asc
Description: PGP signature

Bug#1064824: node-d3: fails to export map and other functions

2024-02-28 Thread Julian Gilbey 
reassign 1064824 node-d3 5.16.0+~cs5.28.10-1
severity 1064824 serious
retitle 1064824 node-d3: fails to export map and probably other functions
thanks

(I'm raising this to severity "serious" because it breaks a different
package.)

(For background to the discovery of this bug, see below.)  The Debian
version of d3.js does not export "map", which it should.  This breaks
the libjs-d3-tip package.  In fact, the export lists are significantly
different:

$ grep '^exports' d3-npm/dist/d3.js | sort > /tmp/d3-npm.exports
$ grep ^exports /usr/share/nodejs/d3/dist/d3.js | sort > /tmp/d3-debian.exports
$ diff -u /tmp/d3-npm.exports /tmp/d3-debian.exports
[520 lines, lots of differences are due to "$1" type suffixes/
$ sed -e 's/\$[0-9]*;/;/' /tmp/d3-npm.exports > /tmp/d3-npm.exports.trimmed
$ sed -e 's/\$[0-9]*;/;/' /tmp/d3-debian.exports > 
/tmp/d3-debian.exports.trimmed
$ diff -u /tmp/d3-npm.exports.trimmed /tmp/d3-debian.exports.trimmed

This gives lots of differences still; stripping down to just the
differences still has many, many differences: some new exports not in
the original d3, and some lost exports; the list begins:

+exports.Adder = Adder;
-exports.bisect = bisectRight;
+exports.bin = bin;
+exports.bisect = bisect;
+exports.bisectCenter = bisectCenter;
+exports.blur2 = blur2;
+exports.blur = blur;
+exports.blurImage = blurImage;
+exports.count = count;
-exports.csvFormatRow = csvFormatRow;
-exports.csvFormatValue = csvFormatValue;

Now, one *might* guess that the reason for this is that d3 imports
lots of d3-* modules, and these have been updated and changed since d3
v5.16.0 was released, so these changes are reflected in the new list
of exports (though people assuming that the Debian version of d3
v5.16.0 reflects the upstream version will then be confused and
possibly frustrated).  But that doesn't explain the absence of map,
which comes from (node-)d3-collection.  For
/usr/share/nodejs/d3-collection/dist/d3-collection.js reads:

exports.entries = entries;
exports.keys = keys;
exports.map = map;
exports.nest = nest;
exports.set = set;
exports.values = values;

So I cannot fathom why map is not being exported by d3 itself.  There
may be other cases of this behaviour, but I have not investigated
further.  I am quite stumped by this.


Background to this: I'm trying to package a new package which provides
a web server to visualise some data.  The package includes a few
precompiled JavaScript libraries obtained from npmjs.com, and the
server works fine with them.  But following Debian policy, I need to
replace those with the source packages.  And the server then doesn't
work.

The JavaScript libraries which the package uses are: d3 v5.16.0,
d3-tip, apparently v0.9.1, along with jQuery and bootstrap4.  I have
replaced all of these with the versions in the corresponding Debian
packages (and I've just uploaded a new version of d3-tip, thinking
that that was the cause of the bug).

When visiting the served web page, the console log gives the error
message:

Uncaught (in promise) TypeError: t.map is not a function
n http://localhost:8080/js/d3/d3-tip.min.js:1
main http://localhost:8080/js/index.js:848
async* http://localhost:8080/js/index.js:993

(This has changed from the original bug report as the minimised new
version of d3-tip has t.map instead of h.map.)

d3-tip.js requires d3-collection, from which it calls a map function.
I tried replacing d3-tip.min.js with the pre-packaged version rather
than the (newly built) Debian version, but that did not help.  I
reverted that change and instead replaced d3.v5.min.js (which is a
copy of /usr/share/nodejs/d3/dist/d3.min.js) with the version provided
by upstream, which is a verbatim copy of the npmjs.com d3.min.js, and
the server then worked perfectly.  So this told me that it is the
Debian compiled d3 which is not working correctly.

Best wishes,

   Julian