Re: [GNC-dev] WebKitGtk now defaults to Gtk4

[david whiting]

Just want to check that I understand the problem correctly. Is there a
reason why we can't send HTML reports to the browser, as suggested,
then print or save to pdf from the browser? Do we need to generate
postscript or pdf directly from gnucash? I know that at the moment
there can be problems with html reports and pagination, in particular,
but there is a solution for that, e.g., using paged.js
https://pagedjs.org/made-with-paged.js.html

I know paged.js from its use in an R package. Here's some examples:
- a report https://slides.yihui.org/2019-ISU-pagedown.html#12
- a poster with a complex layout
https://slides.yihui.org/2019-ISU-pagedown.html#20
- a letter https://slides.yihui.org/2019-ISU-pagedown.html#24
- a journal article in html and pdf side-by-side
https://slides.yihui.org/2019-ISU-pagedown.html#34

Or have I misunderstood the problem?

David

On Tue, 21 Nov 2023 at 03:36, john  wrote:
>
> Adrien,
>
> Sending the HTML to the browser is easy. Writing the reports in PDF directly 
> requires a complete rewrite of the report output code that's hardwired to 
> generate HTML using basic scheme text output, plus replacing the 
> javascript-based charts solution with some other library that generates 
> PostScript vector graphics (https://graphviz.org  comes 
> to mind). That's a pretty big rewrite, probably a solid year of work for an 
> experienced developer with lots of available time and assuming that there's a 
> suitable library to abstract generating the PDF boilerplate.
>
> The reason using the web browser to display the html loses the 
> transaction/account links is because they're implemented using a callback 
> function that we can register with WebKitWebViewGtk. That goes away when the 
> browser isn't part of GnuCash. It's not impossible, of course: It should be 
> possible to embed a bit of javascript that calls back to GnuCash on a (new) 
> socket listener that would do the linked action.
>
> Regards,
> John Ralls
>
>
> > On Nov 19, 2023, at 21:32, Adrien Monteleone 
> >  wrote:
> >
> > Thank you for the explanation John. I seem to recall a bit of that from an 
> > older discussion.
> >
> > While it seems there is no ideal all-in-one solution, might two of your 
> > last three suggestions be doable or is each one a serious pile of work on 
> > its own?
> >
> > That is:
> >
> > *Send the HTML/JS/CSS to the default browser
> > *Write reports directly to PDF
> >
> > The drill-down links I think get lost either way, and maybe that is 
> > unacceptable, but View, Print, PDF export, and get charts. (The writing to 
> > PDF would just be for export)
> >
> > And I'm not quite following the loss of drill-down links on the first 
> > option. Certainly, I can open a local HTML file in my browser and it can 
> > contain links to local resources. Is there a way to expose a URI to a 
> > GnuCash transaction or other resource that might work? Or is the fact that 
> > it has to leave 'GTK land' for 'native land' in the browser and back in 
> > such a case the problem? Or is it that targeting anything within the data 
> > file is the tricky/impossible part? (I admit, while I've used local HTML to 
> > access local files, I've never tried to access only a portion of a file or 
> > even a marker in a file)
> >
> > Apologies if my lack of understanding the guts of it all is noise.
> >
> > Regards,
> > Adrien
> >
> > On 11/19/23 11:20 PM, john wrote:
> >> The reason for selecting WebKitGtk is the Gtk: It had WebKitWebView and 
> >> was supported in most distros and Mingw.
> >> The Mozilla team seem to have stopped maintaining their embedding facility 
> >> around 2015 and it is apparently badly bit-rotted.
> >> Blink is Chromium's rendering engine so that's the same as Sherlock's 
> >> suggested CEF, i.e. Chromium Embedded Framework. It supports Gtk only on 
> >> Linux; on macOS and Windows it build to the native toolkits, so we'd have 
> >> to hack the build to provide Gtk3 on macOS and Windows. There's no mention 
> >> of support for MinGW, so that might not be possible. The level of effort 
> >> needed both to get and keep the ports working might make it impractical 
> >> even if it is possible, but the only way to find that out is to try.
> >> Note that while most repos have a Chromium package, only Arch and Nix have 
> >> CEF packages. Adding it as a GnuCash dependency is likely to get most 
> >> distros to drop us, so nearly everybody on Linux/BSD would be forced to 
> >> use flatpak.
> >
> > ___
> > gnucash-devel mailing list
> > gnucash-devel@gnucash.org
> > https://lists.gnucash.org/mailman/listinfo/gnucash-devel
>
> ___
> gnucash-devel mailing list
> gnucash-devel@gnucash.org
> https://lists.gnucash.org/mailman/listinfo/gnucash-devel



-- 
David Whiting
___
gnucash-devel mailing list
gnucash-devel@gnucash.org

Re: [GNC-dev] WebKitGtk now defaults to Gtk4

[Christopher Lam]

On Sun, 19 Nov 2023 at 12:01, john  wrote:

> I spent some time this afternoon poking at webkit replacement. The options
> I found are:
> * Keep using WebKitGtk
> * Figure out how to wrap a native (meaning Apple AppKit or Microsoft)
> WebView in a GtkWidget
>

^ This one would be my vote if it's possible. As long as we can use an
evergreen browser, it should do the trick.


> * Send the HTML/JS/CSS to the default browser. This will lose the links to
> GnuCash objects like transactions
> * Configure a GtkTextView to understand HTML tags. This doesn't support
> charts, printing, or PDF export.
> * Write reports directly to PDF. There are libraries out there but I
> didn't find any FLOSS ones that mention CSS styling or drawing charts in
> their feature sets.
>
> Any other ideas?
>
> Regards,
> John Ralls
>
___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: [GNC-dev] WebKitGtk now defaults to Gtk4

[Geert Janssens]

Op zondag 19 november 2023 05:00:49 CET schreef john:
> I spent some time this afternoon poking at webkit replacement. The options I
> found are:
> * Keep using WebKitGtk
> * Figure out how to wrap a native (meaning Apple AppKit or Microsoft)
> WebView in a GtkWidget

I agree it's probably a lot of work, we can get some inspiration from 
WxWidgets. It has a 
wxWebView class that uses a different backend on Windows (IE or Edge) and 
WebKit on 
MacOS and Linux.
https://docs.wxwidgets.org/3.2/classwx_web_view.html[1]


> * Send the HTML/JS/CSS to the default browser. This
> will lose the links to GnuCash objects like transactions

Keeping the links as you suggested in another reply in this thread is 
definitely tricky. I have 
been researching this for an unrelated project. Browsers are (rightfully) very 
reluctant to 
let a webpage interface with the local filesystem in any way. The only 
universal method is 
defining a custom handler for the user's default webbrowser. Similar to what we 
do with 
the integrated WebKit instance, though now it has to be installed for the users 
default 
browser, so somewhere on the user's system, outside of GnuCash' direct control. 
I have 
never looked at whether there's a system-wide location to store such handler 
that's read 
and interpreted by all common browsers. I doubt it though.

Next all links in reports we want to be handled by gnucash need to be set up 
such that this 
custom handler is called. This is usually done by using a unique protocol or 
extension to 
the links that are associated with the handler.

Next we need to figure out how the browser can pass this request to a running 
gnucash 
and how this running gnucash can act on it. I think we may have to complete the 
transition to GtkApplication in order to achieve that. We currently have no way 
to pass 
messages to a running gnucash. GtkApplication on the other hand has methods to 
respond to re-entrant requests (that is re-launching gnucash would instead 
delegate the 
parameters as a message to an already running gnucash -- overly simplified 
explanation).

I imagine there will be several additional complications.

These are some I'm thinking of immediately:
* What should we do if the report remains open in the browser, but gnucash is 
closed by 
the user, yet the user clicks on a link in the report. Should that start 
gnucash again ?
* What if the report is open for another gnucash book than the one that's 
currently open 
in gnucash and the user clicks a link ? I think at the minimum the links need 
to hold a 
reference to the book for which the report was generated. But that's only half 
of the 
solution. GnuCash still needs to act on a book mismatch in some way.
* Currently one could have two instances of gnucash running, with a different 
data set 
loaded and each having their own reports open. What if they both now will send 
their 
reports to the user's default browser ? How will the browser know what gnucash 
to 
forward the requests to when a user click on a report link ? Here also the 
migration to 
GtkApplication could help in solving this issue. Particularly if it's 
configured to be a 
singleton application. In this case there could only be a single gnucash 
running accepting 
messages from the browser. As with the earlier issues the links embedded in our 
reports 
would need to have an indication as to the gnucash book they relate to. This 
book 
information could be used by the single gnucash instance to act on the proper 
book.
* That leads to the next design challenge though: currently our backend code 
may not be 
capable of managing two GncSessions in one running gnucash instance. If we 
convert to 
GtkApplication in singleton mode, we would loose the option to have two gnucash 
books 
open at once. To fix that we'd have to rework the engine code to be fully 
capable of 
managing more that a single open Session.

So this avenue may be possible, but will definitely open a big can of worms if 
you ask me...

> * Configure a
> GtkTextView to understand HTML tags. This doesn't support charts, printing,
> or PDF export.

While probably less work, the result would be a huge step back IMO.

> * Write reports directly to PDF. There are libraries out
> there but I didn't find any FLOSS ones that mention CSS styling or drawing
> charts in their feature sets.
> 
I don't know of one either. The more common way is to go via an intermediate 
format, be 
it html or xml or whatever. Pandoc (https://pandoc.org/[2]) comes to mind here 
as it can 
handle plenty of input formats. However it doesn't do pdf generation directly, 
it relies on 
other pdf generators. Several are listed on the page I linked. They may be 
usable directly 
or not.

Regards,

Geert


[1] https://docs.wxwidgets.org/3.2/classwx_web_view.html
[2] https://pandoc.org/
___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: [GNC-dev] WebKitGtk now defaults to Gtk4

[John Ralls]

> On Nov 21, 2023, at 1:13 PM, Geert Janssens  
> wrote:
> 
> Keeping the links as you suggested in another reply in this thread is 
> definitely tricky. I have been researching this for an unrelated project. 
> Browsers are (rightfully) very reluctant to let a webpage interface with the 
> local filesystem in any way. The only universal method is defining a custom 
> handler for the user's default webbrowser. Similar to what we do with the 
> integrated WebKit instance, though now it has to be installed for the users 
> default browser, so somewhere on the user's system, outside of GnuCash' 
> direct control. I have never looked at whether there's a system-wide location 
> to store such handler that's read and interpreted by all common browsers. I 
> doubt it though.
> 

Geert,

I have a different idea that might work for your other project too: Instead of 
embedding links, embed a (java)script that connects to a network socket that 
the source program (GnuCash in our case) advertises and decorate it to look 
like a link. This stack overflow provides an example: 
https://stackoverflow.com/questions/9945693/connecting-client-to-server-using-socket-io

Regards,
John Ralls

___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: [GNC-dev] WebKitGtk now defaults to Gtk4

[John Ralls]

> On Nov 21, 2023, at 1:13 PM, Geert Janssens  
> wrote:
> 
> > * Configure a
> > GtkTextView to understand HTML tags. This doesn't support charts, printing,
> > or PDF export.
> 
> While probably less work, the result would be a huge step back IMO.
> 

Maybe just a step sideways: It actually should support printing, GtkTextView is 
the normal control for handling text documents in Gtk so GtkPrintOperation 
should work. Plus 
https://docs.gtk.org/gtk3/property.PrintOperation.export-filename.html exports 
to PDF. That's still true for Gtk4.

So that leaves charts, which we'd have to draw on a cairo (Gtk3) or OpenGL 
(Gtk4) canvas. Qt has https://doc.qt.io/qt-5/qtcharts-index.html but web 
searches turn up only 
https://gitlab.gnome.org/GNOME/goffice/-/tree/master/goffice/graph?ref_type=heads
 a fork of which we gleefully excised a few years ago. That would be a step or 
two backwards but I guess it would be preferable to writing our own chart 
library. The good news is that it is being maintained, though at 2-3 
commits/year I wouldn't say it's actively maintained. It's not likely to 
migrate to Gtk4 any time soon.

Regards,
John Ralls

___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: [GNC-dev] WebKitGtk now defaults to Gtk4

[Adrien Monteleone]

Thanks John,

If the internal links are indeed possible, I'd think HTML to the browser 
would be the way to go. As David noted, if someone wants a PDF, they can 
save it from there.


But all is not lost with respect to a direct PDF export. That might be 
possible by first generating the HTML, but instead of sending it to the 
Browser for viewing, run it through one of the following first, then to 
disk:


1. https://ekoopmans.github.io/html2pdf.js/
2. https://github.com/spipu/html2pdf/

(the first seems a bit more robust and less limited than the second)

Regards,
Adrien

On 11/20/23 9:35 PM, john wrote:

Sending the HTML to the browser is easy. Writing the reports in PDF directly requires 
a complete rewrite of the report output code that's hardwired to generate HTML using 
basic scheme text output, plus replacing the javascript-based charts solution with 
some other library that generates PostScript vector graphics (https://graphviz.org 
 comes to mind). That's a pretty big rewrite, probably a 
solid year of work for an experienced developer with lots of available time and 
assuming that there's a suitable library to abstract generating the PDF boilerplate.

The reason using the web browser to display the html loses the 
transaction/account links is because they're implemented using a callback 
function that we can register with WebKitWebViewGtk. That goes away when the 
browser isn't part of GnuCash. It's not impossible, of course: It should be 
possible to embed a bit of javascript that calls back to GnuCash on a (new) 
socket listener that would do the linked action.


___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: [GNC-dev] WebKitGtk now defaults to Gtk4

[john]

Adrien,

I haven't looked at those specific recommendations, but a general caution about 
dependencies: They must either be widely supported by Linux distros or small 
and simple enough to add to GnuCash itself in the "borrowed" subdirectory. In 
the latter case it means we've forked the project and own it for any bugs. 

As for internal links: Possible doesn't mean not a ton of work and either 
somebody with the requisite experience materializing to do it or one of the 
current core devs learning how.

Regards,
John Ralls

> On Nov 21, 2023, at 06:55, Adrien Monteleone  
> wrote:
> 
> Thanks John,
> 
> If the internal links are indeed possible, I'd think HTML to the browser 
> would be the way to go. As David noted, if someone wants a PDF, they can save 
> it from there.
> 
> But all is not lost with respect to a direct PDF export. That might be 
> possible by first generating the HTML, but instead of sending it to the 
> Browser for viewing, run it through one of the following first, then to disk:
> 
> 1. https://ekoopmans.github.io/html2pdf.js/
> 2. https://github.com/spipu/html2pdf/
> 
> (the first seems a bit more robust and less limited than the second)
> 
> Regards,
> Adrien
> 
> On 11/20/23 9:35 PM, john wrote:
>> Sending the HTML to the browser is easy. Writing the reports in PDF directly 
>> requires a complete rewrite of the report output code that's hardwired to 
>> generate HTML using basic scheme text output, plus replacing the 
>> javascript-based charts solution with some other library that generates 
>> PostScript vector graphics (https://graphviz.org  
>> comes to mind). That's a pretty big rewrite, probably a solid year of work 
>> for an experienced developer with lots of available time and assuming that 
>> there's a suitable library to abstract generating the PDF boilerplate.
>> The reason using the web browser to display the html loses the 
>> transaction/account links is because they're implemented using a callback 
>> function that we can register with WebKitWebViewGtk. That goes away when the 
>> browser isn't part of GnuCash. It's not impossible, of course: It should be 
>> possible to embed a bit of javascript that calls back to GnuCash on a (new) 
>> socket listener that would do the linked action.
> 
> ___
> gnucash-devel mailing list
> gnucash-devel@gnucash.org
> https://lists.gnucash.org/mailman/listinfo/gnucash-devel

___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: [GNC-dev] WebKitGtk now defaults to Gtk4

[john]

> On Nov 21, 2023, at 02:29, Christopher Lam  wrote:
> 
> 
> 
> On Sun, 19 Nov 2023 at 12:01, john  > wrote:
>> I spent some time this afternoon poking at webkit replacement. The options I 
>> found are:
>> * Keep using WebKitGtk
>> * Figure out how to wrap a native (meaning Apple AppKit or Microsoft) 
>> WebView in a GtkWidget
> 
> ^ This one would be my vote if it's possible. As long as we can use an 
> evergreen browser, it should do the trick.\

Note though that it doesn't stop WebKitGtk from driving the deadline for 
migration to Gtk4.

Regards,
John Ralls


___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel