More than a month ago I posted a correction to the tokenizer documentation to which no one replied:
"The Tokenizer documentation for char_separator tokenizer function states that the default argument for the second template type is "char_traits<char>". This is incorrect. The source code in token_functions.hpp clearly shows that the default argument is "std::basic_string<Char>::traits_type". Could this please be corrected ?" You obviously have more charm than I do since you received some replies. So maybe the above can be added to your corrections below of the tokenizer documentation. Vladimir Prus wrote: > I have a few comments regarding the tokenizer library. > > 1. The documentation says that char_delimiters_separator is default > parameter to 'tokenizer' template, and at the same time says that > 'char_delimiters_separator' is deprecated. I think that's confusing > and default parameter should be changed to 'char-separator'. > > 2. The token interator description is very brief. Specifically, it > does not say what that iterator is usefull for, or when it's > preferrable to direct use of tokenizer. The only way to construct the > iterator is via make_token_iterator function which takes two > interators as arguments. The meaning of those arguments is not > documented. > > Lastly, the usage example > > typedef token_iterator_generator<offset_separator>::type Iter; > Iter beg = make_token_iterator<string>(s.begin(),s.end(),f); > Iter end = make_token_iterator<string>(s.end(),s.end(),f); > for(;beg!=end;++beg){ > > appears to be just longer than tokenizer use: > > typedef tokenizer< offset_separator > tok_t; > tok_t tok(s, f); > for(tok_t::iterator i = tok.begin(); i != tok.end(): ++i) { > > so I *really* wonder what this iterator is for. OTOH, if it could be > used like: > > for(token_iterator< offset_separator > i(s, f), e; i != e; ++i) { > } > > it would be definitely simpler and easier. Is something like this > possible? > > 3. The 'escaped_list_separator' template could have default argument > for the first parameter, "Char". > > 4. I almost always try to use tokenizer when values are separated by > commas. Believe me or not, I'm always confused as to which tokenizer > function to use. This time, I read all docs for char_separator and > only then used escaped_list separator -- which does the work out of > the box. Maybe, a different name, like "csv_with_escapes_separator" > or "extended_csv_separator" would help? > It would make immediately clear what this separator is for. _______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost