Re: GSOC: On-the-fly Code Checking for Vim - Challenges
Congradulations, Birgi! Don't know whether you have looked into anything about Eclipse, but I referred to their documentation and learned a lot there. They also talked about how incremental compiling works: http://help.eclipse.org/help33/index.jsp?topic=/org.eclipse.platform.doc.isv/guide/resAdv_builders.htm There are also a lot of interesting resources there. Good luck! On Apr 20, 7:21 am, Birgi Tamersoy [EMAIL PROTECTED] wrote: Hey, If we are indeed deciding an incremental compiler of C/C++, one of the problem for Vim is how to give the delta of the current file every time an incremental compile instruction is issued. What do you mean with the 'delta' of the current file? If what you mean is only the difference between the current version and previous version, what is the difficulty about getting that? If we really need an incremental compiler, I have a crazy idea to simulate incremental effect using a batch compiler, but don't know whether it works... Take C for example. In C the functions are highly localized, and the idea is to put each function in a separate file, including the same set of declaration, and compile only the modified functions. On the first time the compiler receives a source file, it slices the code into many localized pieces (e.g. functions) and put them into separate files, and create specific makefiles to manage the pieces. When a delta arrives, we just update the corresponding pieces, and run (g)make to update the necessary parts, avoiding looking at the unchanged parts. Again it just sounds better, but I'm not sure whether this would incur a lot of overhead by creating such a lot of files... Interesting idea though your assumption of localized functions is a huge one. Clearly you cannot generalize that. Even with purely localized functions this method will cause other problems related to the place of function definition in the code. (Think about the situation where you use a function without declaring an explicit prototype for it.) I still don't know how dynamic marking up and preview window in Vim works... (really ashamed) But I probably won't have time to look at them before next Monday, as it is the end of the semester now and so many deadlines are in the way... I think dynamic marking should not be a problem since we already have some examples of it. Yep, last couple of weeks are really tough :). Regards, Birgi --~--~-~--~~~---~--~~ You received this message from the vim_dev maillist. For more information, visit http://www.vim.org/maillist.php -~--~~~~--~~--~--~---
Re: GSOC: On-the-fly Code Checking for Vim - Challenges
On Apr 21, 10:09 pm, Lin [EMAIL PROTECTED] wrote: Congradulations, Birgi! Thank you very much Lin :), I am very happy to be involved. Don't know whether you have looked into anything about Eclipse, but I referred to their documentation and learned a lot there. They also talked about how incremental compiling works:http://help.eclipse.org/help33/index.jsp?topic=/org.eclipse.platform There are also a lot of interesting resources there. No, not yet, but I totally agree that Eclipse documentation and code will be very helpful. Good luck! Thanks again, I hope we will get a useful feature at the end. :) Birgi --~--~-~--~~~---~--~~ You received this message from the vim_dev maillist. For more information, visit http://www.vim.org/maillist.php -~--~~~~--~~--~--~---
Re: GSOC: On-the-fly Code Checking for Vim - Challenges
Lin wrote: Hi all, [...] To let the syntax checker see the buffer, I think the solution can only be making copies of related files The Flymake plug-in for Emacs, as already referred to in the first post, seems to be running some reasonable heuristics on finding the related files. But I think we can use something like gcc -MM files to find the dependencies for the currently opened files in the buffers. We restrict our scope to these files, and put all of them into a temp directory to do the check. This method looks a bit more efficient than doing heuristical brute force search around the current dir, but it would miss the errors in the files depending on current files but not opened. Or we should still do some heuristics to make up for this? [...] Why not do a real build in the background? Heuristics and coping would not work for projects with complex makefiles. Besides, if you are doing a real compilation and everything is OK the object files generated as a result of this background build are valuable and it is wasteful to throw them away. I think that it maybe possible to rename an original file, save the buffer into a file with the same name as original and do the build. If there are more than one modified buffer you need to save all of them before building or maybe filter buffers to be saved based on a regex or something. In order to make Vim behave the same on the saved buffers the following scenario can be used (file name for the modified buffer is fileName.cpp): 1. :w fileName.cpp.new 2. :autocmd FileChangedShell fileName.cpp OnTheFlyMake_FileChangedShell() 3. :autocmd FileWritePre fileName.cpp OnTheFlyMake_FileWritePre() 4. :!mv fileName.cpp fileName.cpp.old mv fileName.cpp.new fileName.cpp 5. :exec ! . makeprg 6. Parse make output using 'errorformat'. 7. Optionally update quick fix list. 8. :!mv fileName.cpp.old fileName.cpp 9. :autocmd! FileChangedShell fileName.cpp OnTheFlyMake_FileChangedShell() 10. :autocmd! FileWritePre fileName.cpp OnTheFlyMake_FileWritePre() The OnTheFlyMake_FileChangedShell() function makes Vim ignore the fact that the real disk file was changed. The OnTheFlyMake_FileWritePre() function should abort compilation, undo file renames and save the buffer into the disk file. This was user will be able to save his work even when the background compilation is in progress. Of course steps 2,3,9 and 10 does not need to be performed for each and every file. It maybe more convenient (and maybe faster) to register the functions to be run for all files and to check via a script global list if the action has to be performed for the event. --~--~-~--~~~---~--~~ You received this message from the vim_dev maillist. For more information, visit http://www.vim.org/maillist.php -~--~~~~--~~--~--~---
Re: GSOC: On-the-fly Code Checking for Vim - Challenges
Hello everyone, This is another student who is applying for the on-the-fly code checker project in GSoC. On Apr 14, 2:59 am, [EMAIL PROTECTED] wrote: However, I do am interested in something able to understand C++ constructs in order to built even more powerful code transformations and generations. As such, this GSOC project may initiate the development of the API I'm missing. Hence I'm looking forward to it. Would it be useful to have a mechanism for Vim to do some background job? By using threads etc. Another feature becoming possible with this API would be an on-the-fly tags generator. When SOME threshold is reached, we can call a background thread to refresh the tags, which can be in turn reflected in the tags window (which is supplied in the Taglist plugin). But fine tunes should be given for the threshold. The same problem exists for syntax checker. Would anyone find this useful? Lin --~--~-~--~~~---~--~~ You received this message from the vim_dev maillist. For more information, visit http://www.vim.org/maillist.php -~--~~~~--~~--~--~---
Re: GSOC: On-the-fly Code Checking for Vim - Challenges
Bram Moolenaar wrote: Birgi Tamersoy wrote: Hey everybody, Apparently many students (like me) are interested in this project. So, I decided to start a new thread where we can talk about the challenges in the project and how to solve them. I went over previous posts to include all the related ideas in this new thread. Thanks for starting this discussion. Here are a few initial thoughts about the project: I think *speed* is one of the major concerns. Basically, editing speed should not be affected at all by the code checker. Running a compiler in the background with a temporary copy of the code, parsing the error messages and highlighting the related parts is one of the intuitive solutions (which is also implemented in Emacs Flymake). Although this is a to-the-point solution it will be quite slow according to your project size. *Incremental compiling* can be used to enhance this method. Rather than compiling the whole code every time, the checker can compile only the necessary parts. Right now, I don't know if this is supported by the majority of the compilers or the interpreters. Using a *built-in parser* is solution emphasized by Charles E. Campbell Jr in a related post. A parser would be perfect for syntax- checking, but I think the code checker should also detect errors like a misspelled library name. In my opinion the errors and warnings that are highlighted must be 99% correct. It's very annoying if you get an error for something that is not wrong, or compiling fails with an error but the error was not highlighted. I don't think that a built-in parser will be able to reach this accuracy, except perhaps for some simple languages. There are yacc grammars available for several languages, including C, Matlab, and Lisp. The advantage of this approach is similar to the advantage of the syntax highlighting engine; its not particularly tied to any language. I mostly program in C/Matlab myself, but I know there's plenty of other languages and users out there (python, perl, sh/csh/ksh/zsh/tcsh/..., java, lisp, scheme, ada, matlab, ... apologies to those folks for whom I've missed mentioning their favorites) and tying a major feature of vim to one or two languages is probably not going to be universally appreciated. What I was advocating was the ability to use LALR grammars (read: yacc/bison) so that vim isn't tied to a particular language and, in fact, can be extended to cover future languages. The primary disadvantage is that effectively writing a new yacc/bison is not a small task. I have a real thick book about writing a yacc-like parser somewhere; if anyone's interested, I could look up its ISBN. Probably one could set up pipes to an external yacc or bison and avoid that problem, but that introduces the external sort type of problems people have had until vimgrep. If, in fact, a LALR parser is embedded a future GSOC would involve using it to do more accurate syntax highlighting. To support C++ and other object oriented languages would also involve symbol table handling. Sounds like a huge project to me to make it truly generic. I'm not sure I would like it; especially if I got yammered at for incomplete code. Regards, Chip Campbell --~--~-~--~~~---~--~~ You received this message from the vim_dev maillist. For more information, visit http://www.vim.org/maillist.php -~--~~~~--~~--~--~---
Re: GSOC: On-the-fly Code Checking for Vim - Challenges
Lin [EMAIL PROTECTED] wrote: This is another student who is applying for the on-the-fly code checker project in GSoC. On Apr 14, 2:59 am, [EMAIL PROTECTED] wrote: However, I do am interested in something able to understand C++ constructs in order to built even more powerful code transformations and generations. As such, this GSOC project may initiate the development of the API I'm missing. Hence I'm looking forward to it. Would it be useful to have a mechanism for Vim to do some background job? By using threads etc. It all depends on the job I guess. Another feature becoming possible with this API would be an on-the-fly tags generator. When SOME threshold is reached, we can call a background thread to refresh the tags, which can be in turn reflected in the tags window (which is supplied in the Taglist plugin). But fine tunes should be given for the threshold. The same problem exists for syntax checker. Would anyone find this useful? IMHO, there is no need to run ctags in the background. [I already have a plugin (one of the rare I've never published) that updates my tags file every time a buffer is saved. By update I mean that only the updated file is parsed, and that the tags previously associated to this file are replaced by the new ones. At most, I loose 1/10 s every time I hit F2 (mapped to :update). I won't be surprised to learn that there already are several other plugins that do that on vim.org.] My main concern is more that ctags itself is not a very good solution -- for parsing C++ files. A good solution will need to see the current buffer (we are back to your thread proposal), and do a lot of work to really understand the code. -- Luc Hermitte http://lh-vim.googlecode.com/ http://hermitte.free.fr/vim/ --~--~-~--~~~---~--~~ You received this message from the vim_dev maillist. For more information, visit http://www.vim.org/maillist.php -~--~~~~--~~--~--~---
Re: GSOC: On-the-fly Code Checking for Vim - Challenges
There are yacc grammars available for several languages, including C, Matlab, and Lisp. The advantage of this approach is similar to the advantage of the syntax highlighting engine; its not particularly tied to any language. I mostly program in C/Matlab myself, but I know there's plenty of other languages and users out there (python, perl, sh/csh/ksh/zsh/tcsh/..., java, lisp, scheme, ada, matlab, ... apologies to those folks for whom I've missed mentioning their favorites) and tying a major feature of vim to one or two languages is probably not going to be universally appreciated. What I was advocating was the ability to use LALR grammars (read: yacc/bison) so that vim isn't tied to a particular language and, in fact, can be extended to cover future languages. A language checker has to do more than just grammar checking, you also need type checking, checks if a variable is defined before use, etc. Nico --~--~-~--~~~---~--~~ You received this message from the vim_dev maillist. For more information, visit http://www.vim.org/maillist.php -~--~~~~--~~--~--~---
Re: GSOC: On-the-fly Code Checking for Vim - Challenges
On Apr 14, 11:18 pm, Charles E Campbell Jr [EMAIL PROTECTED] wrote: The primary disadvantage is that effectively writing a new yacc/bison is not a small task. I have a real thick book about writing a yacc-like parser somewhere; if anyone's interested, I could look up its ISBN. The biggest advantage for writing a brand new LALR parser I think is that we can make it incremental. I know something about parsing LALR grammars. They would produce some intermediate table from the grammar definition (called status table?) to guide the parser, so that it operates like a finite automata with a stack equipped. If we have that table we can just store the stack and the parser's state to avoid looking back to previous codes. Then we also need a scanner recognizing different types of words. For this we can seek help from tools like Lex. Is C++ an LALR language? Not sure... Regards, Lin --~--~-~--~~~---~--~~ You received this message from the vim_dev maillist. For more information, visit http://www.vim.org/maillist.php -~--~~~~--~~--~--~---
Re: GSOC: On-the-fly Code Checking for Vim - Challenges
On Monday 14 April 2008, Lin wrote: On Apr 14, 11:18 pm, Charles E Campbell Jr [EMAIL PROTECTED] wrote: The primary disadvantage is that effectively writing a new yacc/bison is not a small task. I have a real thick book about writing a yacc-like parser somewhere; if anyone's interested, I could look up its ISBN. The biggest advantage for writing a brand new LALR parser I think is that we can make it incremental. I know something about parsing LALR grammars. They would produce some intermediate table from the grammar definition (called status table?) to guide the parser, so that it operates like a finite automata with a stack equipped. If we have that table we can just store the stack and the parser's state to avoid looking back to previous codes. Then we also need a scanner recognizing different types of words. For this we can seek help from tools like Lex. Is C++ an LALR language? Not sure... Regards, Lin IMHO not a good idea. You will waste most of the time writing/debugging the parser instead of the Vim part (plus all the time you need to solve problems related to the parser and the grammar). Don't reinvent the wheel when there's one that might suit your need sufficiently well. Writing a parser seems not that hard at first, but how many have you written so far ? It really is not an easy task and that's why yacc/bison is still so popular despite it having some limitations. Plus writing a grammar for C sounds as joyful as driving a glowing steel rod through your private parts, considering all its ambiguities. Marc -- Marc Haisenko Comdasys AG Rüdesheimer Str. 7 80686 München Germany Tel.: +49 (0)89 548 433 321 --~--~-~--~~~---~--~~ You received this message from the vim_dev maillist. For more information, visit http://www.vim.org/maillist.php -~--~~~~--~~--~--~---
Re: GSOC: On-the-fly Code Checking for Vim - Challenges
Birgi Tamersoy wrote: Hey everybody, Apparently many students (like me) are interested in this project. So, I decided to start a new thread where we can talk about the challenges in the project and how to solve them. I went over previous posts to include all the related ideas in this new thread. Thanks for starting this discussion. Here are a few initial thoughts about the project: I think *speed* is one of the major concerns. Basically, editing speed should not be affected at all by the code checker. Running a compiler in the background with a temporary copy of the code, parsing the error messages and highlighting the related parts is one of the intuitive solutions (which is also implemented in Emacs Flymake). Although this is a to-the-point solution it will be quite slow according to your project size. *Incremental compiling* can be used to enhance this method. Rather than compiling the whole code every time, the checker can compile only the necessary parts. Right now, I don't know if this is supported by the majority of the compilers or the interpreters. Using a *built-in parser* is solution emphasized by Charles E. Campbell Jr in a related post. A parser would be perfect for syntax- checking, but I think the code checker should also detect errors like a misspelled library name. In my opinion the errors and warnings that are highlighted must be 99% correct. It's very annoying if you get an error for something that is not wrong, or compiling fails with an error but the error was not highlighted. I don't think that a built-in parser will be able to reach this accuracy, except perhaps for some simple languages. Another concern is to make the checker not *annoying*. I cannot say anything about this because I am never annoyed by the code checkers I had used (like Eclipse). So, it would be beneficial to state what is annoying and what is not. According to these ideas, a checker which executes in different levels of detail may be implemented to address the issue... I really think that by the end of this August we can have an on-the- fly code checker which will be used by a majority of the community. What do you think, what are your concerns? Global warming :-). -- My particular problem is with registry entries, which seem to just accumulate like plastic coffee cups... -- Paul Moore /// Bram Moolenaar -- [EMAIL PROTECTED] -- http://www.Moolenaar.net \\\ ///sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\ \\\download, build and distribute -- http://www.A-A-P.org/// \\\help me help AIDS victims -- http://ICCF-Holland.org/// --~--~-~--~~~---~--~~ You received this message from the vim_dev maillist. For more information, visit http://www.vim.org/maillist.php -~--~~~~--~~--~--~---
Re: GSOC: On-the-fly Code Checking for Vim - Challenges
On Sat, Apr 12, 2008 at 8:11 PM, Birgi Tamersoy [EMAIL PROTECTED] wrote: I really think that by the end of this August we can have an on-the- fly code checker which will be used by a majority of the community. I'm not sure how many people are interested in on-the-fly compilation. I know I'm not: the first thing I do when I have to use Eclipse is to turn off that feature. I find that very annoying when I get errors because I've not finished to write some code. I frequently write incomplete code to get the global structure (e.g., loops, conditions) before completing it. IMO Eclipse's and Vim's communities are different from each other. I understand that Eclipse users like such feature, but maybe that's precisely why they use a tool like Eclipse. That's only my two cents, maybe I'm wrong and I'm just too old-school to appreciate such feature :-) --~--~-~--~~~---~--~~ You received this message from the vim_dev maillist. For more information, visit http://www.vim.org/maillist.php -~--~~~~--~~--~--~---
Re: GSOC: On-the-fly Code Checking for Vim - Challenges
Hey everybody,
I'm not really because I program mainly in C++, and my experience demonstrates
that after the first error it is not uncommon to have the compiler completely
lost. Thus I know this will be a very complex task, and I'm not sure it can be
done.
After your posts I made a small experiment with Eclipse and you are
right, even with the following small example the compiler gave two
wrong error messages:
=== example:
public class test_me {
public static void main(String args[]) {
int birgi = 0;
int damla = 1;
System.out.println(birgi+damla);
for(;;)
=== javac output:
test_me.java:8: illegal start of expression
^
test_me.java:7: ';' expected
for(;;)
^
2 errors
*However* Eclipse could figure out one of the missing brackets and
highlighted the corresponding single bracket. It also failed about
finding the second missing bracket and highlighted the second semi-
colon in the for.
Basically this proves that directly parsing the compiler output will
not give very accurate results, especially when the code has some
syntax errors.
Maybe(!?!?) we can overcome this problem by a kind of multi-step code
checking. First step looks for syntax errors, and second step looks
for compile-time errors that are not syntactic. We may be able to
filter the output of the second step according to the results of the
first step???
Birgi
--~--~-~--~~~---~--~~
You received this message from the vim_dev maillist.
For more information, visit http://www.vim.org/maillist.php
-~--~~~~--~~--~--~---
RE: GSOC: On-the-fly Code Checking for Vim - Challenges
hermitte wrote: I'm not sure how many people are interested in on-the-fly compilation. I'm not really because I program mainly in C++, and my experience demonstrates that after the first error it is not uncommon to have the compiler completely lost. Thus I know this will be a very complex task, and I'm not sure it can be done. However, I do am interested in something able to understand C++ constructs in order to built even more powerful code transformations and generations. As such, this GSOC project may initiate the development of the API I'm missing. Hence I'm looking forward to it. Me too! This project will either never see the light of day, or it will perform a miracle and work in only a slightly irritating fashion. In both cases, it would have been better if the effort had been directed at something more productive. Consider the issue recently raised by Dominique Pelle: People editing C++ code would really appreciate a bit more intelligence applied to the tags feature so that Ctrl-] would try a bit harder to jump to the correct overloaded name. That would be tricky, but more achievable and much more worthwhile. I don't see how the intelligent tags request would be assisted by work from on-the-fly syntax checking because common sense confirms Bram's point that writing a generic syntax checker ain't really going to work (and would be unnecessarily complex). John --~--~-~--~~~---~--~~ You received this message from the vim_dev maillist. For more information, visit http://www.vim.org/maillist.php -~--~~~~--~~--~--~---
RE: GSOC: On-the-fly Code Checking for Vim - Challenges
John Beckett [EMAIL PROTECTED] wrote: Consider the issue recently raised by Dominique Pelle: People editing C++ code would really appreciate a bit more intelligence applied to the tags feature so that Ctrl-] would try a bit harder to jump to the correct overloaded name. That would be tricky, but more achievable and much more worthwhile. In this particular case, the most complex task consists in fetching the true type name of the symbol we are interested in which, as it has been pointed out should be already computed somewhere in omnicppcomplete code. Once we have this information, jumping to the right declaration or definition is a piece of cake with vim 7(.1?) VimL API. (I even have two more applications for this information: have my for(each)-iterator snippet automatically write the container type, and fix my enum to switch generator.) I don't see how the intelligent tags request would be assisted by work from on-the-fly syntax checking because common sense confirms Bram's point that writing a generic syntax checker ain't really going to work (and would be unnecessarily complex). The task ahead is indeed likely to be very complex as each language has a different set of syntax, grammar, idioms, etc. -- Luc Hermitte http://lh-vim.googlecode.com/ http://hermitte.free.fr/vim/ --~--~-~--~~~---~--~~ You received this message from the vim_dev maillist. For more information, visit http://www.vim.org/maillist.php -~--~~~~--~~--~--~---
Re: GSOC: On-the-fly Code Checking for Vim - Challenges
Using a *built-in parser* is solution emphasized by Charles E. Campbell Jr in a related post. A parser would be perfect for syntax- checking, but I think the code checker should also detect errors like a misspelled library name. At least for C, you could use the clang C frontend ( clang.llvm.org ). It's a C frontend that uses a library design, so you should be able to use it in this project. As far as I understand, it can serialize its bytecode, so when the code is compiled for real, the parsing wouldn't have to be redone. If you could manage to create a flexible on-the-fly code checking system with an example code-checker for C based on clang (that'll also provide C++ and Obj-C support), that'd be quite useful. Nico --~--~-~--~~~---~--~~ You received this message from the vim_dev maillist. For more information, visit http://www.vim.org/maillist.php -~--~~~~--~~--~--~---
