Re: syntax highlighting

[Stefan Tatschner]

On 05.03.2016 23:47, Anthony LaTorre wrote:
> Hi all,
> 
> I've recently installed cgit on CentOS 6 from the EPEL repository.
> Unfortunately python3 is not available for CentOS 6. I have a couple
> of questions:

Just use the Python2 version of the script; it is in the Git-Log:

https://git.zx2c4.com/cgit/tree/filters/syntax-highlighting.py?id=ceffeb5d52609a38c5d5d410456d796673fe8461

Stefan
___
CGit mailing list
CGit@lists.zx2c4.com
http://lists.zx2c4.com/mailman/listinfo/cgit

Re: [PATCH v3 2/2] filters: Choose 'trac' theme in pygments

[Stefan Tatschner]

Am 14.01.2014 02:33, schrieb Jason A. Donenfeld:
 Personally, I think the trac colors are a bit ugly. I like pastie
 best. But this is just preference.

I personally also like the 'default' theme it is pretty much the same as
'pastie' but it does not add a background color to strings. This
background color looks really strange with multiline strings. It might
be another solution. If we use 'default' I think we will have to
overwrite the background color of div class=highlight to white.



signature.asc
Description: OpenPGP digital signature
___
CGit mailing list
CGit@lists.zx2c4.com
http://lists.zx2c4.com/mailman/listinfo/cgit

Idle time in project overview

[Stefan Tatschner]

I don't know if it is a bug or a feature but I think on this nice
mailing list I could ask without being shot or mutilated. :)

If I have a git repository with multiple branches and I push to another
branch as 'master' the idle time on the project overview is not updated.
I think this behaviour is not correct. When I push to another branch as
master the project is absolutely not in an idle state. I think the idle
time should be updated when I push to any branch.

Stefan



signature.asc
Description: OpenPGP digital signature
___
CGit mailing list
CGit@lists.zx2c4.com
http://lists.zx2c4.com/mailman/listinfo/cgit

[PATCH v2] filters: Improved syntax-highlighting.py

[Stefan Tatschner]

- Switched back to python2 according to a problem in pygments with python3.
  With the next release of pygments this problem should be fixed.
  Issue see here:
  https://bitbucket.org/birkenfeld/pygments-main/issue/901/problems-with-python3
- Just read the stdin, decode it to utf-8 and ignore unknown signs. This ensures
  that even destroyed files do not cause any errors in the filter.
- Improved language guessing:
  - At first use guess_lexer_for_filename for a better detection of the used
 programming languages (even mixed cases will be detected, e.g. php + html).
  - If nothing was found look if there is a shebang and use guess_lexer.
  - As default/fallback choose TextLexer.
- Using inline CSS instead of this sys.stdout.print() hack.
- Using trac theme for pygments (it is very clean and not intrusive like the
  default or pastie theme).
- I had to fix cgit.css according to a alignment issue with the line-numbers
  table.

Signed-off-by: Stefan Tatschner ste...@sevenbyte.org
---
 cgit.css   |  1 +
 filters/syntax-highlighting.py | 48 +-
 2 files changed, 30 insertions(+), 19 deletions(-)

diff --git a/cgit.css b/cgit.css
index 71b0b9b..ef99b5d 100644
--- a/cgit.css
+++ b/cgit.css
@@ -289,6 +289,7 @@ div#cgit table.blob td.linenumbers {
 
 div#cgit table.blob pre {
padding: 0; margin: 0;
+   line-height: 125%;
 }
 
 div#cgit table.blob td.linenumbers a,
diff --git a/filters/syntax-highlighting.py b/filters/syntax-highlighting.py
index 72d9097..b95baed 100755
--- a/filters/syntax-highlighting.py
+++ b/filters/syntax-highlighting.py
@@ -1,13 +1,16 @@
-#!/usr/bin/env python3
+#!/usr/bin/env python2
 
-# This script uses Pygments and Python3. You must have both installed for this 
to work.
+# This script uses Pygments and Python2. You must have both installed
+# for this to work.
+#
 # http://pygments.org/
 # http://python.org/
 #
-# It may be used with the source-filter or repo.source-filter settings in 
cgitrc.
+# It may be used with the source-filter or repo.source-filter settings
+# in cgitrc.
 #
-# The following environment variables can be used to retrieve the configuration
-# of the repository for which this script is called:
+# The following environment variables can be used to retrieve the
+# configuration of the repository for which this script is called:
 # CGIT_REPO_URL( = repo.url   setting )
 # CGIT_REPO_NAME   ( = repo.name  setting )
 # CGIT_REPO_PATH   ( = repo.path  setting )
@@ -18,22 +21,29 @@
 
 
 import sys
-import cgi
-import codecs
-from pygments.lexers import get_lexer_for_filename
 from pygments import highlight
+from pygments.util import ClassNotFound
+from pygments.lexers import TextLexer
+from pygments.lexers import guess_lexer
+from pygments.lexers import guess_lexer_for_filename
 from pygments.formatters import HtmlFormatter
 
-sys.stdin = codecs.getreader(utf-8)(sys.stdin.detach())
-sys.stdout = codecs.getwriter(utf-8)(sys.stdout.detach())
-doc = sys.stdin.read()
+
+# read stdin and decode to utf-8. ignore any unkown signs.
+data = sys.stdin.read().decode(encoding='utf-8', errors='ignore')
+filename = sys.argv[1]
+formatter = HtmlFormatter(encoding='utf-8', style='trac', noclasses=True)
+
 try:
-   lexer = get_lexer_for_filename(sys.argv[1])
-   formatter = HtmlFormatter(style='pastie')
-   sys.stdout.write(style)
-   sys.stdout.write(formatter.get_style_defs('.highlight'))
-   sys.stdout.write(/style)
+lexer = guess_lexer_for_filename(filename, data, encoding='utf-8')
+except ClassNotFound:
+# check if there is any shebang
+if data[0:2] == '#!':
+lexer = guess_lexer(data, encoding='utf-8')
+else:
+lexer = TextLexer(encoding='utf-8')
+except TypeError:
+lexer = TextLexer(encoding='utf-8')
 
-   highlight(doc, lexer, formatter, sys.stdout)
-except:
-   sys.stdout.write(str(cgi.escape(doc).encode(ascii, 
xmlcharrefreplace), ascii))
+# highlight! :-)
+highlight(data, lexer, formatter, outfile=sys.stdout)
-- 
1.8.5.2

___
CGit mailing list
CGit@lists.zx2c4.com
http://lists.zx2c4.com/mailman/listinfo/cgit

Re: [PATCH v2] filters: Improved syntax-highlighting.py

[Stefan Tatschner]

Am 13.01.2014 15:21, schrieb Jason A. Donenfeld:
 On Mon, Jan 13, 2014 at 12:02 PM, Stefan Tatschner ste...@sevenbyte.org 
 wrote:
 - Using inline CSS instead of this sys.stdout.print() hack.
 
 Please don't do this. Inline CSS makes for much bigger files, and
 there's nothing in the HTML5 spec that forbids us from putting style
 tags there in the first place.

Ok. I will revert that change. I think the best solution would be
putting the style definitions in a seperate file. But I will go with the
former implementation.

 - Using trac theme for pygments (it is very clean and not intrusive like the
   default or pastie theme).
 
 This is probably best as a separate commit, since some folks might complain.

Ok. I will do this in a seperate commit.

 - I had to fix cgit.css according to a alignment issue with the line-numbers
   table.
 
 This is this issue, right?
 https://code.google.com/p/chromium/issues/detail?id=141945
 http://data.zx2c4.com/italics-broken.png
 http://data.zx2c4.com/italics-broken.html
 
 Why does changing the line-height to 125% fix it exactly?

No, that's not the issue. By default 'pygments' adds a pre
style=line-height: 125% tag so I had to change the line-height of the
other column as well. To be honest in my opionion it looks also a bit
better with that kind of expanded line-height.

Stefan




signature.asc
Description: OpenPGP digital signature
___
CGit mailing list
CGit@lists.zx2c4.com
http://lists.zx2c4.com/mailman/listinfo/cgit

Re: [PATCH v2] filters: Improved syntax-highlighting.py

[Stefan Tatschner]

Am 13.01.2014 15:50, schrieb Jason A. Donenfeld:
 On Mon, Jan 13, 2014 at 3:47 PM, Stefan Tatschner ste...@sevenbyte.org 
 wrote:
 No, that's not the issue. By default 'pygments' adds a pre
 style=line-height: 125% tag so I had to change the line-height of the
 other column as well.
 
 Strange. It doesn't do that on the current script. Is this because of
 the inline styles? In which case, since we're reverting to the style
 way of doing things, we can just drop this part too from this commit.

You're right this is because of the inline styles. If we just write the
CSS definitions to stdout it doesn't matter and we could remove this as
well.




signature.asc
Description: OpenPGP digital signature
___
CGit mailing list
CGit@lists.zx2c4.com
http://lists.zx2c4.com/mailman/listinfo/cgit

[PATCH v3 2/2] filters: Choose 'trac' theme in pygments

[Stefan Tatschner]

Using trac theme for pygments. It is very clean and not as
intrusive as the default or pastie theme. Especially I do
not like the the 'pastie' theme very much because of the
very strange illustration of multiline strings (the red
background thing).

Signed-off-by: Stefan Tatschner ste...@sevenbyte.org
---
 filters/syntax-highlighting.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/filters/syntax-highlighting.py b/filters/syntax-highlighting.py
index 53081a4..67855d1 100755
--- a/filters/syntax-highlighting.py
+++ b/filters/syntax-highlighting.py
@@ -32,7 +32,7 @@ from pygments.formatters import HtmlFormatter
 # read stdin and decode to utf-8. ignore any unkown signs.
 data = sys.stdin.read().decode(encoding='utf-8', errors='ignore')
 filename = sys.argv[1]
-formatter = HtmlFormatter(encoding='utf-8', style='pastie')
+formatter = HtmlFormatter(encoding='utf-8', style='trac')
 
 try:
 lexer = guess_lexer_for_filename(filename, data, encoding='utf-8')
-- 
1.8.5.2

___
CGit mailing list
CGit@lists.zx2c4.com
http://lists.zx2c4.com/mailman/listinfo/cgit

[PATCH v3 1/2] filters: Improved syntax-highlighting.py

[Stefan Tatschner]

- Switched back to python2 according to a problem in pygments with python3.
  With the next release of pygments this problem should be fixed.
  Issue see here:
  https://bitbucket.org/birkenfeld/pygments-main/issue/901/problems-with-python3
- Just read the stdin, decode it to utf-8 and ignore unknown signs. This ensures
  that even destroyed files do not cause any errors in the filter.
- Improved language guessing:
  - At first use guess_lexer_for_filename for a better detection of the used
 programming languages (even mixed cases will be detected, e.g. php + html).
  - If nothing was found look if there is a shebang and use guess_lexer.
  - As default/fallback choose TextLexer.

Signed-off-by: Stefan Tatschner ste...@sevenbyte.org
---
 filters/syntax-highlighting.py | 52 +++---
 1 file changed, 33 insertions(+), 19 deletions(-)

diff --git a/filters/syntax-highlighting.py b/filters/syntax-highlighting.py
index 72d9097..53081a4 100755
--- a/filters/syntax-highlighting.py
+++ b/filters/syntax-highlighting.py
@@ -1,13 +1,16 @@
-#!/usr/bin/env python3
+#!/usr/bin/env python2
 
-# This script uses Pygments and Python3. You must have both installed for this 
to work.
+# This script uses Pygments and Python2. You must have both installed
+# for this to work.
+#
 # http://pygments.org/
 # http://python.org/
 #
-# It may be used with the source-filter or repo.source-filter settings in 
cgitrc.
+# It may be used with the source-filter or repo.source-filter settings
+# in cgitrc.
 #
-# The following environment variables can be used to retrieve the configuration
-# of the repository for which this script is called:
+# The following environment variables can be used to retrieve the
+# configuration of the repository for which this script is called:
 # CGIT_REPO_URL( = repo.url   setting )
 # CGIT_REPO_NAME   ( = repo.name  setting )
 # CGIT_REPO_PATH   ( = repo.path  setting )
@@ -18,22 +21,33 @@
 
 
 import sys
-import cgi
-import codecs
-from pygments.lexers import get_lexer_for_filename
 from pygments import highlight
+from pygments.util import ClassNotFound
+from pygments.lexers import TextLexer
+from pygments.lexers import guess_lexer
+from pygments.lexers import guess_lexer_for_filename
 from pygments.formatters import HtmlFormatter
 
-sys.stdin = codecs.getreader(utf-8)(sys.stdin.detach())
-sys.stdout = codecs.getwriter(utf-8)(sys.stdout.detach())
-doc = sys.stdin.read()
+
+# read stdin and decode to utf-8. ignore any unkown signs.
+data = sys.stdin.read().decode(encoding='utf-8', errors='ignore')
+filename = sys.argv[1]
+formatter = HtmlFormatter(encoding='utf-8', style='pastie')
+
 try:
-   lexer = get_lexer_for_filename(sys.argv[1])
-   formatter = HtmlFormatter(style='pastie')
-   sys.stdout.write(style)
-   sys.stdout.write(formatter.get_style_defs('.highlight'))
-   sys.stdout.write(/style)
+lexer = guess_lexer_for_filename(filename, data, encoding='utf-8')
+except ClassNotFound:
+# check if there is any shebang
+if data[0:2] == '#!':
+lexer = guess_lexer(data, encoding='utf-8')
+else:
+lexer = TextLexer(encoding='utf-8')
+except TypeError:
+lexer = TextLexer(encoding='utf-8')
 
-   highlight(doc, lexer, formatter, sys.stdout)
-except:
-   sys.stdout.write(str(cgi.escape(doc).encode(ascii, 
xmlcharrefreplace), ascii))
+# highlight! :-)
+# printout pygments' css definitions as well
+sys.stdout.write('style')
+sys.stdout.write(formatter.get_style_defs('.highlight'))
+sys.stdout.write('/style')
+highlight(data, lexer, formatter, outfile=sys.stdout)
-- 
1.8.5.2

___
CGit mailing list
CGit@lists.zx2c4.com
http://lists.zx2c4.com/mailman/listinfo/cgit

Re: [PATCH v3 1/2] filters: Improved syntax-highlighting.py

[Stefan Tatschner]

Am 13.01.2014 22:50, schrieb Jason A. Donenfeld:
 Perfect! Applied. Thanks for going through all the revisions.

thanks as well. :)
I have one simple question (just out of curiosity; I do not want to bash
anybody).

Why did you apply my patch with tabs instead of spaces? I was wondering
because I adjusted the python script according to pep8 [1] and I'm sure
the patchfile was with spaces. Maybe you have an automatic convert script?

[1] http://www.python.org/dev/peps/pep-0008/#tabs-or-spaces



signature.asc
Description: OpenPGP digital signature
___
CGit mailing list
CGit@lists.zx2c4.com
http://lists.zx2c4.com/mailman/listinfo/cgit

Re: [PATCH v3 1/2] filters: Improved syntax-highlighting.py

[Stefan Tatschner]

Am 13.01.2014 23:16, schrieb Jason A. Donenfeld:
 On Mon, Jan 13, 2014 at 11:13 PM, Stefan Tatschner ste...@sevenbyte.org 
 wrote:
 Why did you apply my patch with tabs instead of spaces? I was wondering
 because I adjusted the python script according to pep8 [1] and I'm sure
 the patchfile was with spaces. Maybe you have an automatic convert script?
 
 I like tabs. They're for tabbing. There's a character for doing the
 thing that they do. I like to use that character. The rest of cgit is
 the same way. So, python scripts in the tree follow suit.

got it. :)




signature.asc
Description: OpenPGP digital signature
___
CGit mailing list
CGit@lists.zx2c4.com
http://lists.zx2c4.com/mailman/listinfo/cgit

Re: RESEND: syntax-highlighting.py

[Stefan Tatschner]

Am 08.01.2014 16:19, schrieb Jason A. Donenfeld:
 Okay reading this closer, it seems what the one in-tree could benefit from is:
 
 - Expanded list of filename mappings, made more generic than what you
 have in your script, but basically the same idea. -- { pkgbuild:
 bashlexer, cmakelists.txt, cmakelexer }. Is there a way to do
 this that ties directly into pygment's guess_lexer_for_filename? If
 not, could you submit a patch upstream?

I have created an upstream pull request and it got merged after 30
seconds [1]. Moreover it seems like the cmakelists.txt thing was also
fixed [2].

Should I remove the filename mapping thing in my patch according to
upstream or should I keep this until the next pygments release?


[1]
https://bitbucket.org/birkenfeld/pygments-main/commits/7cc1e6d0143445302445360fb8b963a7644f1912
[2]
https://bitbucket.org/birkenfeld/pygments-main/src/8fda165f5da04bdf5040fc7b8d3e1589d8fc1b4f/pygments/lexers/text.py?at=default#cl-1609



signature.asc
Description: OpenPGP digital signature
___
CGit mailing list
CGit@lists.zx2c4.com
http://lists.zx2c4.com/mailman/listinfo/cgit