Re: [webkit-dev] WebKit render tree

[Andy Somogyi]

What I’m thinking about is what if all of my code could be used as a webkit 
extension, i.e. I could make the class definitions conform and create idl files 
for each of my class, and stick them all in a new directory under 
WebCore/Modules. Looks like then, I would have to modify a single CMake file 
(CMake is cool, I’ve been using CMake for over a decade :) )

One thing that’s not entirely clear to me is how exactly is JSC made aware of 
the all of the generated class bindings. The internals of each of the binding 
files is very clear: a series of accessor methods, and references to all these 
are made in a JSC::ClassInfo struct. I see a createPrototype static method in 
each class, but I can’t seem to find out how these get called. 

I’d like to be able to create native windows (either NSWindow on OS X, or 
native W64 via CreateWindow on Windows), and I’d like to create a set of C++ 
objects with appropriate JavaScript bindings that currently executing 
JavaScript can create and interact with. Its trivial for me to create JSC 
bindings, the bit that I’m not sure about is should these live in the host 
application side, or in the new modules I’d create under WebCore/Modules. 

I’m looking closer at the internal of JavaScriptcore, and remember, I’m working 
on a new programming language. It looks like it would be straightforward to 
compile it to the JavaSciriptCore binary object layout. This would create a 
bridge where all of the new object could interact directly with existing JS 
objects. 




> On Jun 1, 2016, at 4:09 PM, Myles C. Maxfield  wrote:
> 
> From the description, it sounds like you have some native code which needs to 
> be able to modify the DOM and you need events from the DOM to be able to call 
> code from your library. You can do this today without modifying WebKit.
> 
> The way for native code to modify the DOM is for the native code to generate 
> a JavaScript string and submit the string to be run inside the JavaScript 
> context of the page. You can do this in WK1 by JSEvaluateScript(), the 
> WebScriptObject, or by stringByEvaluatingJavaScriptFromString. In WebKit2, 
> WKWebView has a method evaluateJavaScript (Or from C you can run 
> WKPageRunJavaScriptInMainFrame, but this isn’t considered API)
> 
> You can also use Objective-C DOM bindings to manipulate the DOM directly from 
> native code, this approach is discouraged.
> 
> One way for DOM events to call code from your library is to create a 
> JavaScript object or method which is backed by native code. Then, the 
> JavaScript event handler can call the functions on this object, which causes 
> your native code to run. This can be done with the functions listed inside 
> Source/JavaScriptCore/API/JSObjectRef.h.
> 
> In WebKit2, this native-backed-object can still be made, but it must be done 
> inside the Web process (not the UI process), and therefore must be done with 
> the InjectedBundle. Inside the InjectedBundle, you can run native code in the 
> web process, and there are facilities to send messages between the Web 
> process & UI process. You can find out more if you search for 
> “InjectedBundle” inside Source/WebKit2/UIProcess/API. Note, however, that use 
> of the InjectedBundle is not considered API.
> 
> We rely on the above mechanisms heavily in our testing infrastructure. Our 
> Tools/DumpRenderTree and Tools/WebKitTestRunner projects use the techniques 
> described above.
> 
> The Windows port currently doesn’t support WebKit2, so you’ll have to use the 
> WebKit1 approach on Windows.
> 
> 
>> On Jun 1, 2016, at 11:49 AM, Andy Somogyi > > wrote:
>> 
>> 
>>> On Jun 1, 2016, at 1:37 PM, Myles C. Maxfield >> > wrote:
>>> 
>>> Replies inline.
>> 
>>> Generating a DOM is usually done in JavaScript. I'm curious about the 
>>> problem you are trying to solve where JavaScript is not sufficient.
>>> 
>>> Drawing into an existing window is usually done by inserting a WebView into 
>>> the view 
>>> reallyhierarchy of the window. Yet again, I'm interested in why this is not 
>>> sufficient.
>> 
>> I have the source code of the new language, and a parser/compiler for it 
>> written in C++. I’d hate to have to introduce a new set of languages into 
>> the mix. 
>> 
>> The new programming language is designed to be visually edited, hence the 
>> need for hit detection. So, a component may be dragged from one region to 
>> another. This would trigger an event which would cause one branch of the AST 
>> to be pruned, modified and re-attached to another branch. This in turn would 
>> trigger a re-gen of the render tree, and eventually a repaint. I've done 
>> something very similar in the past where I created a visual computer algebra 
>> system. The entire runtime, including AST are written in C++, and it would 
>> be nice to connect it to an existing rendering/layout engine, and be able to 
>> 

Re: [webkit-dev] WebKit render tree

[Myles C. Maxfield]

From the description, it sounds like you have some native code which needs to 
be able to modify the DOM and you need events from the DOM to be able to call 
code from your library. You can do this today without modifying WebKit.

The way for native code to modify the DOM is for the native code to generate a 
JavaScript string and submit the string to be run inside the JavaScript context 
of the page. You can do this in WK1 by JSEvaluateScript(), the WebScriptObject, 
or by stringByEvaluatingJavaScriptFromString. In WebKit2, WKWebView has a 
method evaluateJavaScript (Or from C you can run 
WKPageRunJavaScriptInMainFrame, but this isn’t considered API)

You can also use Objective-C DOM bindings to manipulate the DOM directly from 
native code, this approach is discouraged.

One way for DOM events to call code from your library is to create a JavaScript 
object or method which is backed by native code. Then, the JavaScript event 
handler can call the functions on this object, which causes your native code to 
run. This can be done with the functions listed inside 
Source/JavaScriptCore/API/JSObjectRef.h.

In WebKit2, this native-backed-object can still be made, but it must be done 
inside the Web process (not the UI process), and therefore must be done with 
the InjectedBundle. Inside the InjectedBundle, you can run native code in the 
web process, and there are facilities to send messages between the Web process 
& UI process. You can find out more if you search for “InjectedBundle” inside 
Source/WebKit2/UIProcess/API. Note, however, that use of the InjectedBundle is 
not considered API.

We rely on the above mechanisms heavily in our testing infrastructure. Our 
Tools/DumpRenderTree and Tools/WebKitTestRunner projects use the techniques 
described above.

The Windows port currently doesn’t support WebKit2, so you’ll have to use the 
WebKit1 approach on Windows.


> On Jun 1, 2016, at 11:49 AM, Andy Somogyi  wrote:
> 
> 
>> On Jun 1, 2016, at 1:37 PM, Myles C. Maxfield > > wrote:
>> 
>> Replies inline.
> 
>> Generating a DOM is usually done in JavaScript. I'm curious about the 
>> problem you are trying to solve where JavaScript is not sufficient.
>> 
>> Drawing into an existing window is usually done by inserting a WebView into 
>> the view 
>> reallyhierarchy of the window. Yet again, I'm interested in why this is not 
>> sufficient.
> 
> I have the source code of the new language, and a parser/compiler for it 
> written in C++. I’d hate to have to introduce a new set of languages into the 
> mix. 
> 
> The new programming language is designed to be visually edited, hence the 
> need for hit detection. So, a component may be dragged from one region to 
> another. This would trigger an event which would cause one branch of the AST 
> to be pruned, modified and re-attached to another branch. This in turn would 
> trigger a re-gen of the render tree, and eventually a repaint. I've done 
> something very similar in the past where I created a visual computer algebra 
> system. The entire runtime, including AST are written in C++, and it would be 
> nice to connect it to an existing rendering/layout engine, and be able to 
> respond to user events directly in my C++ code. 
> 
> I see this as a very dynamic application, and all of the internal logic which 
> is a mix of C and JIT compiled code from the new language needs to interact 
> with both the visual representation, and the physics engine (the Lang is 
> designed for real-time physics simulation). Ideally, I'd like to be able to 
> render a visual representation of a language element to an OpenGL texture, 
> and have this exist in the physics engine. 
> 
> However, being as the DOM is publicly available from WebKit in C++ (I think 
> they’re all exposed publicly as pure virtual interfaces), it would be 
> relatively straightforward for me to to compile the new language to target 
> the C++ v-table ABI. 
> 
> 
>>> Do you think this would be possible using WebCore as a library and have 
>>> these custom render tree / dom tree derived objects live in my own library 
>>> (this is really, really the way I’d prefer to do things), or do you think 
>>> it would be better to add these objects directly into the WebCore library. 
>>> 
>> 
>> Are you planning on committing any changes to trunk?
>> 
>> Of course it is ::possible:: to do something like this - it's all software. 
>> Are you asking if it's easy to do it? Or the best way to do it
> 
> On OSX, I don’t think I’ll have to do any changes at all to WebKit. I’ve got 
> proof of concept working where I create a new RenderElement derived object, 
> and give it a new GraphicsContext. 
> 
> However, on Windows, the classes I use from WebKit in my own code would need 
> to be exported via the WEBCORE_EXPORT macro, this would need to be added to 
> each class definition in WebKit. I suppose what I could do is write a quick 
> script which 

Re: [webkit-dev] WebKit render tree

[Andy Somogyi]

> On Jun 1, 2016, at 1:37 PM, Myles C. Maxfield  wrote:
> 
> Replies inline.

> Generating a DOM is usually done in JavaScript. I'm curious about the problem 
> you are trying to solve where JavaScript is not sufficient.
> 
> Drawing into an existing window is usually done by inserting a WebView into 
> the view hierarchy of the window. Yet again, I'm interested in why this is 
> not sufficient.

I have the source code of the new language, and a parser/compiler for it 
written in C++. I’d really hate to have to introduce a new set of languages 
into the mix. 

The new programming language is designed to be visually edited, hence the need 
for hit detection. So, a component may be dragged from one region to another. 
This would trigger an event which would cause one branch of the AST to be 
pruned, modified and re-attached to another branch. This in turn would trigger 
a re-gen of the render tree, and eventually a repaint. I've done something very 
similar in the past where I created a visual computer algebra system. The 
entire runtime, including AST are written in C++, and it would be nice to 
connect it to an existing rendering/layout engine, and be able to respond to 
user events directly in my C++ code. 

I see this as a very dynamic application, and all of the internal logic which 
is a mix of C and JIT compiled code from the new language needs to interact 
with both the visual representation, and the physics engine (the Lang is 
designed for real-time physics simulation). Ideally, I'd like to be able to 
render a visual representation of a language element to an OpenGL texture, and 
have this exist in the physics engine. 

However, being as the DOM is publicly available from WebKit in C++ (I think 
they’re all exposed publicly as pure virtual interfaces), it would be 
relatively straightforward for me to to compile the new language to target the 
C++ v-table ABI. 


>> Do you think this would be possible using WebCore as a library and have 
>> these custom render tree / dom tree derived objects live in my own library 
>> (this is really, really the way I’d prefer to do things), or do you think it 
>> would be better to add these objects directly into the WebCore library. 
>> 
> 
> Are you planning on committing any changes to trunk?
> 
> Of course it is ::possible:: to do something like this - it's all software. 
> Are you asking if it's easy to do it? Or the best way to do it

On OSX, I don’t think I’ll have to do any changes at all to WebKit. I’ve got 
proof of concept working where I create a new RenderElement derived object, and 
give it a new GraphicsContext. 

However, on Windows, the classes I use from WebKit in my own code would need to 
be exported via the WEBCORE_EXPORT macro, this would need to be added to each 
class definition in WebKit. I suppose what I could do is write a quick script 
which adds this macro to each class that I use as part of my build process. 
That way, I would not have to touch the upstream webkit source at all. 

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] WebKit render tree

[Myles C. Maxfield]

Replies inline.

> On May 31, 2016, at 11:19 PM, Andy Somogyi  wrote:
> 
> Hi,
> 
> I’m a programming language researcher, and we are working on a new visual 
> programming language. 
> 
> I’m investigating using WebCore as the rendering component of our language 
> editing/visualization system. 
> 
> Essentially what I’d like to be able to do is to programmatically (C++) 
> generate 1: a DOM tree of a new family of Element derived object, 2: 
> rendering tree, 3: layout the tree, 4: attach a GraphicsContext to an 
> existing window or bitmapped resource, and 5: use WebCore’s GraphicsContext 
> to render the custom rendering tree. I’d also like to create a family of 
> custom Element derived objects that would be attached to the rendering tree 
> and use the rendering tree. I’d also like to respond to the standard events, 
> i.e. mouse, keyboard, etc…

Generating a DOM is usually done in JavaScript. I'm curious about the problem 
you are trying to solve where JavaScript is not sufficient.

Drawing into an existing window is usually done by inserting a WebView into the 
view hierarchy of the window. Yet again, I'm interested in why this is not 
sufficient.

> 
> Do you think this would be possible using WebCore as a library and have these 
> custom render tree / dom tree derived objects live in my own library (this is 
> really, really the way I’d prefer to do things), or do you think it would be 
> better to add these objects directly into the WebCore library. 
> 

Are you planning on committing any changes to trunk?

Of course it is ::possible:: to do something like this - it's all software. Are 
you asking if it's easy to do it? Or the best way to do it?

> I don’t think accessing any of the render tree objects in webcore would cause 
> any issues on Mac/Unix, however, Windows sadly requires those EXPORT macros 
> on each class def. If I added these, what would be the WebKit policy of 
> accepting these changes? 
> 
> What is the, for lack of a better word, the “viability” of WebKit on Windows? 
> Our project fundamentally has to be cross-platform, and currently, I’m not 
> aware of any webkit based browsers on Windows. I’m currently looking into 
> using webkit, firefox or blink. I’ve essentially eliminated blink because 
> their code is very hard to follow, much harder than webkit or firefox, and I 
> think the webkit code is the easiest to understand and use. Webkit is 
> currently my first choice, but my only hesitation is will webkit continued to 
> be supported on Windows. 

Windows is an official port, just like iOS / Mac / EFL / GTK.

> 
> thanks
> 
> -- Andy Somogyi PhD
> School of Informatics and Computing
> Indiana University
> ___
> webkit-dev mailing list
> webkit-dev@lists.webkit.org
> https://lists.webkit.org/mailman/listinfo/webkit-dev
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] WebKit render tree

[Konstantin Tokarev]

01.06.2016, 09:19, "Andy Somogyi" :
> Hi,
>
> I’m a programming language researcher, and we are working on a new visual 
> programming language.
>
> I’m investigating using WebCore as the rendering component of our language 
> editing/visualization system.
>
> Essentially what I’d like to be able to do is to programmatically (C++) 
> generate 1: a DOM tree of a new family of Element derived object, 2: 
> rendering tree, 3: layout the tree, 4: attach a GraphicsContext to an 
> existing window or bitmapped resource, and 5: use WebCore’s GraphicsContext 
> to render the custom rendering tree. I’d also like to create a family of 
> custom Element derived objects that would be attached to the rendering tree 
> and use the rendering tree. I’d also like to respond to the standard events, 
> i.e. mouse, keyboard, etc…

Are you really sure you need to go that deep?

Can't your goals be achieved with just custom DOM tree + style sheets + drawing 
that on exisisting resource using standard WebCore rendering machinery?

If this is the case I propose you to take a look at QtWebKit port, where you 
can build DOM in C++ and use custom QPainter/QPaintEngine for low-level drawing 
operations. Qt also provides you a way to simulate user input events like key 
presses or mouse moves, or you can filter out or override real input events.

Here is my fork which uses updated WebKit from 2016:

https://github.com/annulen/webkit
(See https://github.com/annulen/webkit/wiki for more details)

>
> Do you think this would be possible using WebCore as a library and have these 
> custom render tree / dom tree derived objects live in my own library (this is 
> really, really the way I’d prefer to do things), or do you think it would be 
> better to add these objects directly into the WebCore library.
>
> I don’t think accessing any of the render tree objects in webcore would cause 
> any issues on Mac/Unix, however, Windows sadly requires those EXPORT macros 
> on each class def. If I added these, what would be the WebKit policy of 
> accepting these changes?

I don't think exporting private APIs will be accepted, and I don't think it 
makes much sense because they may be changed at any time anyway. If you need 
them, you will be safer with your customized WebKit fork.


>
> What is the, for lack of a better word, the “viability” of WebKit on Windows? 
> Our project fundamentally has to be cross-platform, and currently, I’m not 
> aware of any webkit based browsers on Windows. I’m currently looking into 
> using webkit, firefox or blink. I’ve essentially eliminated blink because 
> their code is very hard to follow, much harder than webkit or firefox, and I 
> think the webkit code is the easiest to understand and use. Webkit is 
> currently my first choice, but my only hesitation is will webkit continued to 
> be supported on Windows.

AFAIK, there is no indication that Windows support is going to be dropped 
anytime soon.

>
> thanks
>
> -- Andy Somogyi PhD
> School of Informatics and Computing
> Indiana University
> ,
>
> ___
> webkit-dev mailing list
> webkit-dev@lists.webkit.org
> https://lists.webkit.org/mailman/listinfo/webkit-dev


-- 
Regards,
Konstantin
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

[webkit-dev] WebKit render tree

[Andy Somogyi]

Hi,

I’m a programming language researcher, and we are working on a new visual 
programming language. 

I’m investigating using WebCore as the rendering component of our language 
editing/visualization system. 

Essentially what I’d like to be able to do is to programmatically (C++) 
generate 1: a DOM tree of a new family of Element derived object, 2: rendering 
tree, 3: layout the tree, 4: attach a GraphicsContext to an existing window or 
bitmapped resource, and 5: use WebCore’s GraphicsContext to render the custom 
rendering tree. I’d also like to create a family of custom Element derived 
objects that would be attached to the rendering tree and use the rendering 
tree. I’d also like to respond to the standard events, i.e. mouse, keyboard, 
etc…

Do you think this would be possible using WebCore as a library and have these 
custom render tree / dom tree derived objects live in my own library (this is 
really, really the way I’d prefer to do things), or do you think it would be 
better to add these objects directly into the WebCore library. 

I don’t think accessing any of the render tree objects in webcore would cause 
any issues on Mac/Unix, however, Windows sadly requires those EXPORT macros on 
each class def. If I added these, what would be the WebKit policy of accepting 
these changes? 

What is the, for lack of a better word, the “viability” of WebKit on Windows? 
Our project fundamentally has to be cross-platform, and currently, I’m not 
aware of any webkit based browsers on Windows. I’m currently looking into using 
webkit, firefox or blink. I’ve essentially eliminated blink because their code 
is very hard to follow, much harder than webkit or firefox, and I think the 
webkit code is the easiest to understand and use. Webkit is currently my first 
choice, but my only hesitation is will webkit continued to be supported on 
Windows. 

thanks

-- Andy Somogyi PhD
School of Informatics and Computing
Indiana University___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] Webkit render tree dump

[David Hyatt]

I actually have a giant set of changes that need to be made to the  
render tree dumping... many of which are outlined in comments.  The  
problem is it causes all the test results to have to be updated.


Right now many silly and incorrect things are being dumped right now  
just for backwards compatibility.  It's not ideal.


dave

On Dec 1, 2009, at 1:14 PM, Darin Adler wrote:


On Nov 30, 2009, at 6:29 PM, pundarik rajkhowa wrote:

Thanks for the info. I am aware of the pixel test, but for my  
purpose that cant be used, since text-based comparision is  
preferred. Regarding the style attributes, the webkit dump actually  
contains a few of them like color, bgcolor, borderstyle etc. Is  
there any specific reason for that ? Also if I dump other  
attributes like font-weight, text-decoration etc using the style()  
api of the RenderObject, will that help catching bugs specific to  
these attributes ?


It would be OK to dump more. There are two difficulties:

   1) If we add more to what is dumped, the additional information  
may create even more noise in existing test results, repeated things  
that make the salient details hard to find. The original choice of  
what to dump was based on a tradeoff in a hope to make the dumps  
readable.


   2) If we add more to what is dumped, that patch has to update the  
results of thousands of tests on multiple platforms.


Sometimes you can come up with a rule to decide when to dump that  
can mitigate both problems (1) and (2), for example, dumping only  
when values are unusual.


   -- Darin

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev


___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

Re: [webkit-dev] Webkit render tree dump

[Darin Adler]

On Nov 18, 2009, at 8:55 PM, pundarik rajkhowa wrote:

 I have a doubt regarding the content of webkit render tree dump. In this 
 dump, do we only print the HTML node information(body, div etc) or we print 
 the style information also(font-weight, text-decoration etc). From what I 
 have seen, it looks like only node information is printed. In such a case how 
 a bug is caught if it is related to style attributes ? 

Some bugs related to style attributes affect the layout of the web page, and so 
are detected by the change in size of the rendered elements. But other bugs 
related to style can only be caught by explicit queries of style by JavaScript 
code, or with the “pixel tests”, test results that use a graphical dump of a 
fixed-size browser window.

-- Darin

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

[webkit-dev] Webkit render tree dump

[pundarik rajkhowa]

Hi,

I have a doubt regarding the content of webkit render tree dump. In this
dump, do we only print the HTML node information(body, div etc) or we print
the style information also(font-weight, text-decoration etc). From what I
have seen, it looks like only node information is printed. In such a case
how a bug is caught if it is related to style attributes ?

regards,
Pundarik
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev