----- On Jun 3, 2017, at 3:45 PM, Philippe Proulx <eeppelitel...@gmail.com> wrote:
> On Fri, Jun 2, 2017 at 10:24 PM, craig harmer < [ mailto:charms...@gmail.com | > charms...@gmail.com ] > wrote: >> hi all, >> the suggestion to write a pre-processor for LTTng tracepoints is interesting >> and >> may be a good idea. however, i'm not sure that it will provide a general >> purpose solution. >> the problem i for see is that tracepoint invocations, at least, should be >> written in the syntax of the language they're being used in (a programmer >> would >> would expect the tracepoint() function call to look like any other function >> call in the language). >> so the tracepoint parser needs to understand the function invocation syntax >> of >> each language. it also needs to understand at least some of the rest of the >> syntax of the language -- in the case of Go it at least needs to be able >> recognize "import" statements and import those files and it needs to >> understand >> when something that looks like a tracepoint() is actually embedded in a >> comment >> or string. it also needs to emit code that matches the syntax of the >> language. >> so a general purpose lttng tracepoint parsing tool would need at least a >> limited >> understanding of the syntax and grammar of each language for reading the >> source >> file in and then writing the source file out. there could be code in the >> middle >> that's common across languages, but probably not a lot. perhaps an lttng >> tracepoint parsing tool could be better structured as a common framework for >> language specific tracepoint parsers to be plugged in to. >> but writing a parser that actually understands the grammar of a language is >> not >> easy. if i was really going to do this, i'd be tempted to see if i could just >> take this from gcc (which has front-ends that supports C, C++, Go, and other >> languages (i think)). >> and then there's the tracepoint definitions. they could be written using the >> barectf() tool that Philippe mentions, but i'm not sure if that would be >> satisfactory for most programmers, who would prefer to write tracepoint >> definitions in the same language that they are programming in. they would >> also >> need to support the data types of the language (i'm thinking Python here as >> an >> example, where tracepoints would be expected to support most of the Python >> basic data types). > There's a little misunderstanding here. > The idea is to have a universal tracepoint definition language (and what I > suggested is that a subset of CTF 2, of which the syntax is JSON (or YAML > which > translates to JSON directly), could be used for this) which translates to > functions, classes, macros, etc. in selected programming languages. Depending > on the destination language, we can generate serialization functions directly > or wrappers which use LTTng-UST behind the scenes. So in your case, this tool > would generate Go source files which would contain public functions which > correspond to tracepoints. We never considered parsing an existing programming > language to infer tracepoints. > Of course this requires the developer to define tracepoints in this universal > language. That's an additional developing step needed to allow the developer > to > use tracepoints in his/her favorite language. But there are other advantages, > like generating tracepoints in two different languages from the same > tracepoint > definitions. Like Mathieu wrote, it's the same spirit as Flex and Bison. With > Bison, you don't use the C preprocessor to write your grammar rules: the tool > reads a Bison input file and generates a parser source file. Just like barectf > reads a YAML configuration file which defines stream and event classes and > generates tracer source files. The problem is that this is often not enough, and we sometimes need to have pre/post code run before/after the serialization to prepare the arguments. This happens for instance in lttng-modules with the LTTNG_TRACEPOINT_EVENT_CODE() macro. Ideally we should find a way to glue together this high-level description and language-native helper code. Thoughts ? Thanks, Mathieu > Phil >> because Go seems to be similar to C i'm strongly tempted to just co-opt the C >> pre-processor and existing LTTng tracepoint machinery and modify the LTTng >> header files to generate valid Go code for a tracepoint definition. this does >> add a LTTng tracepoint tool to the steps required to compile Go, except that >> the tracepoint tool is called "cpp" (plus header files). >> i don't really know Go yet and i hae not been assigned to get LTTng to work >> in >> Go, so this is all quite preliminary. but i think the project i'm about to >> start on really needs LTTng or LTTng-like tracepoint functionality so i'm >> motivated to see if this can work. >> i'm going to investigate some more. >> --craig >> On Fri, Jun 2, 2017 at 10:05 AM, Philippe Proulx < [ >> mailto:eeppelitel...@gmail.com | eeppelitel...@gmail.com ] > wrote: >>> On Fri, Jun 2, 2017 at 11:58 AM, Mathieu Desnoyers < [ >>> mailto:mathieu.desnoy...@efficios.com | mathieu.desnoy...@efficios.com ] > >>> wrote: >>>> ----- On Jun 2, 2017, at 2:27 AM, craig harmer < [ >>>> mailto:charms...@gmail.com | >>>> charms...@gmail.com ] > wrote: >>>>> hi all, >>>>> has anyone looked at porting LTTng to [ >>>>> https://en.wikipedia.org/wiki/Go_%28programming_language%29 | Go ] (AKA [ >>>>> https://golang.org/ | Golang ] )? >>>> Not yet ! I'm glad someone is looking into it. :) >>>>> Go is a language very similar to C, but it has intrinsic support for >>>>> concurrency >>>>> and better memory protection (at the expense of including garbage >>>>> collection). >>>>> its from Google and is pretty hot these days for big data and "web scale" >>>>> distributed systems. the similarity to C is not surprising since two of >>>>> the >>>>> three authors were also involved in the development of UNIX in at AT&T in >>>>> the >>>>> 1970's: Rob Pike and Ken Thompson (Robert Griesemer is the third author). >>>>> Go is able to link with C object files, so implementing support for >>>>> tracef(3) >>>>> should not be very difficult. >>>>> but i want to have full support for user-defined tracepoint() events. >>>>> that looks >>>>> to be much trickier since the LTTng tracepoint() functionality relies >>>>> heavily >>>>> on (some might say abuses >>> "abuses" is appropriate ;-) >>>>> ) the C pre-processor, and Go does not have an equivalent of the C >>>>> pre-processor. however, i don't see an obvious reason why Go source files >>>>> that >>>>> contain tracepoints couldn't be run through the C pre-processor -- except >>>>> that >>>>> Go specific LTTng header files would be required. >>>>> [ https://tour.golang.org/welcome/1 | here is an example of "hello, >>>>> world" in Go >>>>> ] . (note the use of "import" rather then "#include"). >>>>> i've spent more than a few hours looking at the C code generated by the C >>>>> pre-processor trying to track down exactly why a tracepoint() definition >>>>> is >>>>> causing a compiler error, so i think i have an idea of just how hard it >>>>> would >>>>> be to development equivalent header files for Go + C pre-processor. it >>>>> may be >>>>> the case that the Go "header files", once developed, would be compiler >>>>> specific, i.e only work with [ https://golang.org/doc/install/gccgo | >>>>> gccgo ] >>>>> (the gcc front-end for Go). >>>>> anyway, i'm wondering if anybody has attempted this and/or what your >>>>> thoughts >>>>> would be. >>>> In the case of C/C++ code, using the C preprocessor got the job done >>>> without >>>> requiring any additional >>>> dependency. Given the context you describe, perhaps it would be wise to >>>> consider >>>> introducing a dedicated >>>> "lttng probe description" parser. The idea here would be to parse the >>>> tracepoint >>>> probe definitions and >>>> translate those into native code for various languages (e.g. Golang). >>>> This would create a new lttng utility that would be required to to produce >>>> the >>>> language-specific files from >>>> those descriptions. I would allow us to port the "tracepoint" concept to >>>> many >>>> more languages easily, and >>>> not be so much tied to the C preprocessor anymore. >>>> So perhaps adding this new tool as a dependency that needs to be invoked >>>> prior >>>> to compilation might >>>> not be too much of an issue ? I see it as being slightly similar to the >>>> role >>>> accomplished by Flex and >>>> Bison: they are required to translate from source files to an intermediary >>>> language, and then the >>>> resulting files can be included into the distribution source packages, so >>>> only >>>> those who aim at >>>> changing the source descriptions need to have the translation tool >>>> installed. >>>> Thoughts ? >>> Good point and we thought about this before. In fact this is just what >>> barectf < >>> [ http://barectf.org/ | http://barectf.org/ ] > does, for example. What I >>> see >>> in the future is that a subset of (eventual) CTF 2's metadata language >>> (JSON) >>> could be used here (or an equivalent, human-friendly YAML, like barectf) to >>> describe LTTng event and stream classes and translate them to what's needed >>> to >>> record such events for a given programming language. CTF 2's user attributes >>> can be used to insert additional, language-specific properties, like >>> parameter >>> types and names, and other options. >>> Phil >>>> Thanks, >>>> Mathieu >>>>> --craig >>>>> _______________________________________________ >>>>> lttng-dev mailing list >>>>> [ mailto:lttng-dev@lists.lttng.org | lttng-dev@lists.lttng.org ] >>>>> [ https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev | >>>>> https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev ] >>>> -- >>>> Mathieu Desnoyers >>>> EfficiOS Inc. >>>> [ http://www.efficios.com/ | http://www.efficios.com ] >>>> _______________________________________________ >>>> lttng-dev mailing list >>>> [ mailto:lttng-dev@lists.lttng.org | lttng-dev@lists.lttng.org ] >>>> [ https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev | >>>> https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev ] -- Mathieu Desnoyers EfficiOS Inc. http://www.efficios.com
_______________________________________________ lttng-dev mailing list lttng-dev@lists.lttng.org https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev