Re: [Mesa-dev] [PATCH 0/4] RFC: Quadratic probing for the win

I've looked into Mesa hash-table some time ago, didn't finish anything (sigh, I have troubles finishing things off :( ). Anyway some things I got from there, in no particular order: 1. I've created a tool that records hash query traces and then replays them, sorta like apitrace for hashing.

Re: [Mesa-dev] [PATCH 01/12] glcpp: Print preprocessor output to string_buffer

19.01.2017 01:46, Ian Romanick пишет: On 01/07/2017 11:02 AM, Vladislav Egorov wrote: glcpp's printing is an obvious low hanging fruit: 1. It unnecessarily uses formatted printing to print output of preprocessing. To print just one character '+' it first uses vsnprintf(&qu

Re: [Mesa-dev] [PATCH] util: import sha1 implementation from OpenBSD

16.01.2017 16:13, Emil Velikov пишет: Hi Vladislav, On 14 January 2017 at 01:50, Vladislav Egorov <vegorov...@gmail.com> wrote: 14.01.2017 01:45, Timothy Arceri пишет: I'm asking for a chance to test before we jump in, its probably not a big deal and I may even still be able to red

Re: [Mesa-dev] [PATCH 4/4] glsl: Use hash table cloning in copy propagation

14.01.2017 02:32, Connor Abbott пишет: On Fri, Jan 13, 2017 at 1:55 PM, Thomas Helland <thomashellan...@gmail.com> wrote: 2017-01-13 18:41 GMT+01:00 Vladislav Egorov <vegorov...@gmail.com>: 13.01.2017 15:31, Tapani Pälli пишет: On 01/12/2017 09:23 PM, Thomas Helland wro

Re: [Mesa-dev] [PATCH] util: import sha1 implementation from OpenBSD

14.01.2017 01:45, Timothy Arceri пишет: I'm asking for a chance to test before we jump in, its probably not a big deal and I may even still be able to reduce my use of hashing but it would be nice to be given a few days to test and even explore alternatives before jumping on this implementation.

Re: [Mesa-dev] [PATCH] util: import sha1 implementation from OpenBSD

14.01.2017 00:17, Matt Turner пишет: On Fri, Jan 13, 2017 at 1:01 PM, Vladislav Egorov <vegorov...@gmail.com> wrote: 2017-01-13 22:43 GMT+03:00 Emil Velikov <emil.l.veli...@gmail.com>: On 13 January 2017 at 19:22, Vladislav Egorov <vegorov...@gmail.com> wrote: 13.01.2017 1

Re: [Mesa-dev] [PATCH] util: import sha1 implementation from OpenBSD

2017-01-13 22:43 GMT+03:00 Emil Velikov <emil.l.veli...@gmail.com>: > > On 13 January 2017 at 19:22, Vladislav Egorov <vegorov...@gmail.com> wrote: > > 13.01.2017 19:51, Emil Velikov пишет: > >> > >> From: Emil Velikov <emil.veli...@collabora.com>

Re: [Mesa-dev] [PATCH] util: import sha1 implementation from OpenBSD

13.01.2017 19:51, Emil Velikov пишет: From: Emil Velikov At the moment we support 5+ different implementations each with varying amount of bugs - from thread safely problems [1], to outright broken implementation(s) [2] In order to accommodate these we have 150+

Re: [Mesa-dev] [PATCH 4/4] glsl: Use hash table cloning in copy propagation

13.01.2017 15:31, Tapani Pälli пишет: On 01/12/2017 09:23 PM, Thomas Helland wrote: Walking the whole hash table, inserting entries by hashing them first is just a really really bad idea. We can simply memcpy the whole thing. Maybe it is just 'really' not 'really really' since I don't spot

Re: [Mesa-dev] [PATCH 3/4] util: Implement a hash table cloning function

12.01.2017 22:23, Thomas Helland пишет: --- src/util/hash_table.c | 22 ++ src/util/hash_table.h | 2 ++ 2 files changed, 24 insertions(+) diff --git a/src/util/hash_table.c b/src/util/hash_table.c index 9e643af8b2..702f465382 100644 --- a/src/util/hash_table.c +++

Re: [Mesa-dev] [PATCH 01/12] glcpp: Print preprocessor output to string_buffer

08.01.2017 17:05, Grazvydas Ignotas пишет: Some spelling suggestions, since you seem to be doing another version of these patches: Yes, the fuzzer uncovered many bugs and deviations in behaviour from the old flex/bison path. Trivial, but numerous. So I need some time to hunt all the problem

Re: [Mesa-dev] [PATCH 11/12] glcpp: Create fast path hand-written scanner

08.01.2017 17:11, Grazvydas Ignotas пишет: On Sat, Jan 7, 2017 at 9:02 PM, Vladislav Egorov <vegorov...@gmail.com> wrote: @@ -582,6 +609,333 @@ HEXADECIMAL_INTEGER 0[xX][0-9a-fA-F]+[uU]? %% +static void +glcpp_fast_skip_singleline_comment (glcpp_parser_t *parser, char *

Re: [Mesa-dev] [PATCH 08/12] glcpp: Use strpbrk in the line continuations pass

, sadly. Will try other fuzzers as well. 08.01.2017 00:56, Vladislav Egorov пишет: Oh. Thanks! This is embarrassing. I have no idea how did I manage to miss it. Line continuations are so rare that only two games in my shader-db use them -- Talos Principle and Serious Sam (both use Serious Engine

Re: [Mesa-dev] [PATCH 08/12] glcpp: Use strpbrk in the line continuations pass

or something. I'll send fixup. 07.01.2017 22:53, Gustaw Smolarczyk пишет: 7 sty 2017 20:04 "Vladislav Egorov" <vegorov...@gmail.com <mailto:vegorov...@gmail.com>> napisał(a): To find newlines the line continuations removal pass uses strchr() twice -- first time to

[Mesa-dev] [PATCH 06/12] ralloc: Use strnlen() inside of strncat()

If the str is long or isn't null-terminated, strlen() could take a lot of time or even crash. I don't know why was it used in the first place, maybe for platforms without strnlen(), but strnlen() is already used inside of ralloc_strndup(), so this change should not additionally break anything. ---

[Mesa-dev] [PATCH 11/12] glcpp: Create fast path hand-written scanner

At this point up to ~80% of preprocessing time is spent in generated parser/lexer functions, so it's not possible to improve speed further without making changes to lexer and parser. In most of the cases all complex machinery of tokenizing, parsing and printing is completely unnecessary. On most

[Mesa-dev] [PATCH 10/12] glcpp/tests: Allow different trailing whitespace

There is no point really to enforce exact amount of trailing whitespace in preprocessor output. Other preprocessors generate even less trailing whitespace than glcpp. The less trailing whitespace, the better. --- src/compiler/glsl/glcpp/tests/glcpp-test | 4 ++-- 1 file changed, 2 insertions(+),

[Mesa-dev] [PATCH 04/12] glcpp: Use string_buffer for continuations removal

Migrate removal of line continuations to string_buffer. Before this it used ralloc_strncat() to append strings, which internally each time calculates strlen() of its argument. Its argument is entire shader, so it multiple time scans the whole shader text. --- src/compiler/glsl/glcpp/glcpp.h | 11

[Mesa-dev] [PATCH 12/12] glcpp: Substitute trivial macros in the fast path

The vast majority of macros are trivial, simple numbers or identifiers. Substitute them in the fast path, no need to bailout for them to the slow path. --- src/compiler/glsl/glcpp/glcpp-lex.l | 77 +++-- 1 file changed, 74 insertions(+), 3 deletions(-) diff --git

[Mesa-dev] [PATCH 09/12] glcpp: Avoid unnecessary linear_strdup

The lexer copies every identifier it finds. At first look it seems needless, because all these identifiers are already available stored inside of shader text, but my experiments with immutable explicitly sized strings (ptr, len) resulted in a lot of code change and didn't result in any speed

[Mesa-dev] [PATCH 03/12] glcpp: Use Bloom filter before identifier search

Absolute majority of identifiers in shaders are not macro references. Many shaders don't use preprocessing macros at all. Almost all queries to parser->defines hash-table will be unsuccessful. Note that all predefined macros start either with "GL_" or with "__". Moreover, even user-defined macros

[Mesa-dev] [PATCH 02/12] glcpp: Avoid unnecessary strcmp()

strcmp() is slow. Initiate comparison with "__LINE__" or "__FILE__" only if the identifier starts with '_', which is rare. --- src/compiler/glsl/glcpp/glcpp-parse.y | 14 +- 1 file changed, 9 insertions(+), 5 deletions(-) diff --git a/src/compiler/glsl/glcpp/glcpp-parse.y

[Mesa-dev] [PATCH 01/12] glcpp: Print preprocessor output to string_buffer

glcpp's printing is an obvious low hanging fruit: 1. It unnecessarily uses formatted printing to print output of preprocessing. To print just one character '+' it first uses vsnprintf("%s", "+") to calculate number of characters in the formatted string (while it's known statically), then

[Mesa-dev] [PATCH 05/12] ralloc: Avoid calling vsnprintf() twice

Typical perf output of ralloc's printf-like functions looks like that: 0,14% ralloc_vasprintf_rewrite_tail 0,06% printf_length 0,05% __vsnprintf_chk 0,02% resize 0,01% __vsnprintf_chk@plt They spend most of the time inside of libc's formatted i/o functions they call twice

[Mesa-dev] [PATCH 08/12] glcpp: Use strpbrk in the line continuations pass

To find newlines the line continuations removal pass uses strchr() twice -- first time to find '\n', second time to find '\r'. Now, if the shader uses Unix line separators '\n', the file could contain no '\r' at all, so each time it will scan to the end of the shader. Use strpbrk() standard

[Mesa-dev] [PATCH 07/12] glcpp: Skip unnecessary line continuations removal

Overwhelming majority of shaders don't use line continuations. In my shader-db only shaders from the Talos Principle and Serious Sam used them, less than 1% out of all shaders. Optimize for this case, don't do any copying if no line continuation was found. --- src/compiler/glsl/glcpp/pp.c | 10

[Mesa-dev] [PATCH 00/12] Improve GLSL preprocessor performance

't really matter for preprocessor. Other preprocessors drop trailing whitespace entirely. Vladislav Egorov (12): glcpp: Print preprocessor output to string_buffer glcpp: Avoid unnecessary strcmp() glcpp: Use Bloom filter before identifier search glcpp: Use string_buffer for continuations removal rallo

Re: [Mesa-dev] [PATCH 06/14] util: Make a copy of the existing hash table implementation

I think this data structure is known as multimap (in STL it's named std::unordered_multimap, in Scala it's MultiMap, in Apache Commons it's MultiValueMap and so on). https://en.wikipedia.org/wiki/Multimap 01.01.2017 21:37, Thomas Helland пишет: --- src/util/Makefile.sources | 2

Re: [Mesa-dev] [PATCH 2/7] glsl/glcpp: use ralloc_sprint_rewrite_tail to avoid slow vsprintf

01.01.2017 06:41, Kenneth Graunke пишет: On Sunday, January 1, 2017 1:34:27 AM PST Marek Olšák wrote: From: Marek Olšák This reduces compile times by 4.5% with the Gallium noop driver and gl_constants::GLSLOptimizeConservatively == true. Compile times of...what

Re: [Mesa-dev] [PATCH 1/2] glsl: Don't rehash the values when copying to new table

:53 AM, Vladislav Egorov wrote: I've looked into it recently (I'm working on series of many various trivial optimizations) and it's faster to just memcpy() it. Just throwing out superfluous hashing still keeps slow hash-table insertion around -- with resizing, rehashing, memory allocation/deallo

Re: [Mesa-dev] [PATCH 1/2] glsl: Don't rehash the values when copying to new table

I've looked into it recently (I'm working on series of many various trivial optimizations) and it's faster to just memcpy() it. Just throwing out superfluous hashing still keeps slow hash-table insertion around -- with resizing, rehashing, memory allocation/deallocation, internal

Re: [Mesa-dev] [PATCH 1/2] glsl: add driconf to zero-init unintialized vars

Maybe it would make sense to add browsers like Firefox or Chrome to this list? I noticed that many shaders on e.g. ShaderToy expect zero-initialisation and fail miserably on Mesa. It seems that on most browsers/platforms/drivers variables are zero-initialised, so authors don't see anything