On Mon, 2015-11-02 at 14:14 -0500, David Malcolm wrote:
> On Fri, 2015-10-30 at 00:15 -0600, Jeff Law wrote:
> > On 10/23/2015 02:41 PM, David Malcolm wrote:
> > > As in the previous version of this patch
> > >   "Implement tree expression tracking in C FE (v2)"
> > > the patch now captures ranges for all C expressions during parsing within
> > > a new field of c_expr, and for all tree nodes with a location_t, it stores
> > > them in ad-hoc locations for later use.
> > >
> > > Hence compound expressions get ranges; see:
> > >    
> > > https://dmalcolm.fedorapeople.org/gcc/2015-09-22/diagnostic-test-expressions-1.html
> > >
> > > and for this example:
> > >
> > >    int test (int foo)
> > >    {
> > >      return foo * 100;
> > >             ^^^   ^^^
> > >    }
> > >
> > > we have access to the ranges of "foo" and "100" during C parsing via
> > > the c_expr, but once we have GENERIC, all we have is a VAR_DECL and an
> > > INTEGER_CST (the former's location is in at the top of the
> > > function, and the latter has no location).
> > >
> > > gcc/ChangeLog:
> > >   * Makefile.in (OBJS): Add gcc-rich-location.o.
> > >   * gcc-rich-location.c: New file.
> > >   * gcc-rich-location.h: New file.
> > >   * print-tree.c (print_node): Print any source range information.
> > >   * tree.c (set_source_range): New functions.
> > >   * tree.h (CAN_HAVE_RANGE_P): New.
> > >   (EXPR_LOCATION_RANGE): New.
> > >   (EXPR_HAS_RANGE): New.
> > >   (get_expr_source_range): New inline function.
> > >   (DECL_LOCATION_RANGE): New.
> > >   (set_source_range): New decls.
> > >   (get_decl_source_range): New inline function.
> > >
> > > gcc/c-family/ChangeLog:
> > >   * c-common.c (c_fully_fold_internal): Capture existing souce_range,
> > >   and store it on the result.
> > >
> > > gcc/c/ChangeLog:
> > >   * c-parser.c (set_c_expr_source_range): New functions.
> > >   (c_token::get_range): New method.
> > >   (c_token::get_finish): New method.
> > >   (c_parser_expr_no_commas): Call set_c_expr_source_range on the ret
> > >   based on the range from the start of the LHS to the end of the
> > >   RHS.
> > >   (c_parser_conditional_expression): Likewise, based on the range
> > >   from the start of the cond.value to the end of exp2.value.
> > >   (c_parser_binary_expression): Call set_c_expr_source_range on
> > >   the stack values for TRUTH_ANDIF_EXPR and TRUTH_ORIF_EXPR.
> > >   (c_parser_cast_expression): Call set_c_expr_source_range on ret
> > >   based on the cast_loc through to the end of the expr.
> > >   (c_parser_unary_expression): Likewise, based on the
> > >   op_loc through to the end of op.
> > >   (c_parser_sizeof_expression) Likewise, based on the start of the
> > >   sizeof token through to either the closing paren or the end of
> > >   expr.
> > >   (c_parser_postfix_expression): Likewise, using the token range,
> > >   or from the open paren through to the close paren for
> > >   parenthesized expressions.
> > >   (c_parser_postfix_expression_after_primary): Likewise, for
> > >   various kinds of expression.
> > >   * c-tree.h (struct c_expr): Add field "src_range".
> > >   (c_expr::get_start): New method.
> > >   (c_expr::get_finish): New method.
> > >   (set_c_expr_source_range): New decls.
> > >   * c-typeck.c (parser_build_unary_op): Call set_c_expr_source_range
> > >   on ret for prefix unary ops.
> > >   (parser_build_binary_op): Likewise, running from the start of
> > >   arg1.value through to the end of arg2.value.
> > >
> > > gcc/testsuite/ChangeLog:
> > >   * gcc.dg/plugin/diagnostic-test-expressions-1.c: New file.
> > >   * gcc.dg/plugin/diagnostic_plugin_test_tree_expression_range.c:
> > >   New file.
> > >   * gcc.dg/plugin/plugin.exp (plugin_test_list): Add
> > >   diagnostic_plugin_test_tree_expression_range.c and
> > >   diagnostic-test-expressions-1.c.
> > 
> > >   /* Initialization routine for this file.  */
> > >
> > > @@ -6085,6 +6112,9 @@ c_parser_expr_no_commas (c_parser *parser, struct 
> > > c_expr *after,
> > >     ret.value = build_modify_expr (op_location, lhs.value, 
> > > lhs.original_type,
> > >                                    code, exp_location, rhs.value,
> > >                                    rhs.original_type);
> > > +  set_c_expr_source_range (&ret,
> > > +                    lhs.get_start (),
> > > +                    rhs.get_finish ());
> > One line if it fits.
> > 
> > 
> > > @@ -6198,6 +6232,9 @@ c_parser_conditional_expression (c_parser *parser, 
> > > struct c_expr *after,
> > >                              ? t1
> > >                              : NULL);
> > >       }
> > > +  set_c_expr_source_range (&ret,
> > > +                    start,
> > > +                    exp2.get_finish ());
> > Here too.
> > 
> > > @@ -6522,6 +6564,10 @@ c_parser_cast_expression (c_parser *parser, struct 
> > > c_expr *after)
> > >           expr = convert_lvalue_to_rvalue (expr_loc, expr, true, true);
> > >         }
> > >         ret.value = c_cast_expr (cast_loc, type_name, expr.value);
> > > +      if (ret.value && expr.value)
> > > + set_c_expr_source_range (&ret,
> > > +                          cast_loc,
> > > +                          expr.get_finish ());
> > And here?
> > 
> > With the nits fixed, this is OK.
> > 
> > I think that covers this iteration of the rich location work and that 
> > you'll continue working with Jason on extending this into the C++ front-end.
> 
> Here's a summary of the current status of this work [1]:
> 
> Patches 1-4 of the kit: these Jeff has approved, with some pre-approved
> nit fixes in 4.  I see these as relatively low risk, and plan to commit
> these today/tomorrow.
> 
> Patches 5-10: Jeff approved these also (again with some nits). These
> feel higher-risk to me, owing to the potential for performance
> regressions; I haven't yet answered at least one of Richi's performance
> questions (impact on time taken to generate the C++ PCH file); the last
> performance testing I did can be seen here:
> https://gcc.gnu.org/ml/gcc-patches/2015-10/msg02283.html
> where the right-most column is this kit.
> 
> CCing Richi to keep him in the loop for the above.  Richi, is there any
> other specific testing you'd want me to do for this?
> Or is it OK to commit, and to see what impact it has on your daily
> performance testing?  (and to revert if the impact is unacceptable).
> 
> Talking about risks: the reduction of the space for ordinary maps by a
> factor of 32, by taking up 5 bits for the packed range information
> optimization (patch 10):
>  https://gcc.gnu.org/ml/gcc-patches/2015-10/msg02539.html
> CCing Dodji: Dodji; is this reasonable?
> 
> I did some experiments back in July on this; instrumented builds of
> openldap and openssl to see how much space we have in location_t:
> https://dmalcolm.fedorapeople.org/gcc/2015-07-23/openldap.csv
> https://dmalcolm.fedorapeople.org/gcc/2015-07-24/openldap.csv
> 
> (these are space-separated:
>            SRPM name
>            sourcefile
>            maximal ordinary location_t
>            minimal macro location_t)
> 
> openldap build #files: 906
> maximal ordinary location_t was:
> sourcefile='/builddir/build/BUILD/openldap-2.4.40/openldap-2.4.40/servers/slapd/bconfig.c'
>           max_ordinary_location=0x0081bd1b
>           (and min_macro_location=0x7ffe5903
> minimal macro location_t was:
> sourcefile='/builddir/build/BUILD/openldap-2.4.40/openldap-2.4.40/servers/slapd/aclparse.c'
>           min_macro_location=0x7ffe57e2
>           (with max_ordinary_location=0x00719775)
> 
> openssl-1.0.1k-8.fc22.src.rpm.x86_64:
>       #files: 1495
> max_ordinary_location=0x00be3726
>  (openssl-1.0.1k/apps/s_client.c)
>  with min_macro_location=0x7ffe7b6b
> 
> min_macro_location=0x7ffdf069 
>  (openssl-1.0.1k/apps/speed.c)
>  with max_ordinary_location=0x00a1abdf
> 
> In all of the above cases, we had enough room to do the bit-packing
> optimization, but this is just two projects (albeit real-world C code).
> 
> Comparing the gap between maximal ordinary map location and minimal
> macro map location, and seeing how much we can expand the ordinary map
> locations, the openldap build had:
>   (0x7ffe57e2 - 0x0081bd1b) / 0x0081bd1b  == factor of 251 i.e.
> 7 bits of space available
> 
> openssl build had:
>   (0x7ffdf069 - 0x00be3726) / 0x00be3726  == factor of 171 i.e. 7 bits
> of space available
> 
> hence allocating 5 bits to packing ranges is (I hope) reasonable.
> 
> 
> Jeff: I'm working on expression ranges in the C++ FE; is that a
> prerequisite for patches 5-10, or can 5-10 go ahead without the C++
> work?  (assuming the other issues above are acceptable).
> 
> Hope this all makes sense and sounds sane
> Dave
> 
> [1] Together the kit gives us:
> * patch 4: infrastructure for printing underlines in
> diagnostic_show_locus and for multiple ranges
> * patches 5-10: the "source_location" (aka location_t) type becomes a
> caret plus a range; the tokens coming from libcpp gain ranges, so
> everything using libcpp gains at least underlines of tokens; the C
> frontend generates sane ranges for expressions as it parses them, better
> showing the user how the parser "sees" their code.
> 
> Hence we ought to get underlined ranges for many diagnostics in C and C
> ++ with this (e.g. input_location gives an underline covering the range
> of the token starting at the caret).  The "caret" should remain
> unchanged from the status quo, so e.g. debugging locations shouldn't be
> affected by the addition of ranges.
> 
> I'm anticipating that we'd need some followup patches to pick better
> ranges for some diagnostics, analogous to the way we convert "warning"
> to "warning_at" for where input_location isn't the best location; I'd
> expect these followup patches to be relative simple and low-risk.

I've addressed the nits raised by Jeff in review, combined patches 5-10
plus "libcpp: add examples to source_location description" into one
patch, verified bootstrap&regrtest (x86_64-pc-linux-gnu) and committed
it to trunk as r230331.

Patch attached for reference.

(I have a separate patch for the C++ FE which bootstraps, but it shows 4
testsuite regressions; hope to fix that today)

>From 89d6a22e3c7d9e670eec48f614368b986a2d0132 Mon Sep 17 00:00:00 2001
From: David Malcolm <dmalc...@redhat.com>
Date: Fri, 23 Oct 2015 12:52:13 -0400
Subject: [PATCH] Source range tracking in libcpp and C FE, with bit-packing

This patch combines:
  [PATCH 05/10] Add ranges to libcpp tokens (via ad-hoc data, unoptimized)
  [PATCH 06/10] Track expression ranges in C frontend
  [PATCH 07/10] Add plugin to recursively dump the source-ranges in a tree (v2)
  [PATCH 08/10] Wire things up so that libcpp users get token underlines
  [PATCH 09/10] Delay some resolution of ad-hoc locations, preserving ranges
  [PATCH 10/10] Compress short ranges into source_location
  [PATCH] libcpp: add examples to source_location description
along with fixes for the nits identified by Jeff.

In particular:
  -const unsigned int LINE_MAP_MAX_COLUMN_NUMBER = (1U << 17);
  +const unsigned int LINE_MAP_MAX_COLUMN_NUMBER = (1U << 12);

rather than to:
  +const unsigned int LINE_MAP_MAX_COLUMN_NUMBER = (1U << 9);

gcc/ChangeLog:
	* Makefile.in (OBJS): Add gcc-rich-location.o.
	* diagnostic.c (diagnostic_append_note): Pass line_table to
	rich_location ctor.
	(emit_diagnostic): Likewise.
	(inform): Likewise.
	(inform_n): Likewise.
	(warning): Likewise.
	(warning_at): Likewise.
	(warning_n): Likewise.
	(pedwarn): Likewise.
	(permerror): Likewise.
	(error): Likewise.
	(error_n): Likewise.
	(error_at): Likewise.
	(sorry): Likewise.
	(fatal_error): Likewise.
	(internal_error): Likewise.
	(internal_error_no_backtrace): Likewise.
	(source_range::debug): Likewise.
	* gcc-rich-location.c: New file.
	* gcc-rich-location.h: New file.
	* genmatch.c (fatal_at): Pass line_table to rich_location ctor.
	(warning_at): Likewise.
	* gimple.h (gimple_set_block): Use set_block function.
	* input.c (dump_line_table_statistics): Dump stats on how many
	ranges were optimized vs how many needed ad-hoc table.
	(write_digit_row): Add "map" param; use its range_bits
	to calculate the per-character offset.
	(dump_location_info): Print the range and column bits for each
	ordinary map.  Use the range bits to calculate the per-character
	offset.  Pass the map as a new param to the various calls to
	write_digit_row.  Eliminate uses of
	ORDINARY_MAP_NUMBER_OF_COLUMN_BITS.
	* print-tree.c (print_node): Print any source range information.
	* rtl-error.c (diagnostic_for_asm): Likewise.
	* toplev.c (general_init): Initialize line_table's
	default_range_bits.
	* tree-cfg.c (move_block_to_fn): Likewise.
	(move_block_to_fn): Likewise.
	* tree-inline.c (copy_phis_for_bb): Likewise.
	* tree.c (tree_set_block): Likewise.
	(get_pure_location): New function.
	(set_source_range): New functions.
	(set_block): New function.
	(set_source_range): New functions.
	* tree.h (CAN_HAVE_RANGE_P): New.
	(EXPR_LOCATION_RANGE): New.
	(EXPR_HAS_RANGE): New.
	(get_expr_source_range): New inline function.
	(DECL_LOCATION_RANGE): New.
	(set_source_range): New decls.
	(get_decl_source_range): New inline function.

gcc/ada/ChangeLog:
	* gcc-interface/trans.c (Sloc_to_locus): Add line_table param when
	calling linemap_position_for_line_and_column.

gcc/c-family/ChangeLog:
	* c-common.c (c_fully_fold_internal): Capture existing souce_range,
	and store it on the result.
	* c-opts.c (c_common_init_options): Set
	global_dc->colorize_source_p.

gcc/c/ChangeLog:
	* c-decl.c (warn_defaults_to): Pass line_table to
	rich_location ctor.
	* c-errors.c (pedwarn_c99): Likewise.
	(pedwarn_c90): Likewise.
	* c-parser.c (set_c_expr_source_range): New functions.
	(c_token::get_range): New method.
	(c_token::get_finish): New method.
	(c_parser_expr_no_commas): Call set_c_expr_source_range on the ret
	based on the range from the start of the LHS to the end of the
	RHS.
	(c_parser_conditional_expression): Likewise, based on the range
	from the start of the cond.value to the end of exp2.value.
	(c_parser_binary_expression): Call set_c_expr_source_range on
	the stack values for TRUTH_ANDIF_EXPR and TRUTH_ORIF_EXPR.
	(c_parser_cast_expression): Call set_c_expr_source_range on ret
	based on the cast_loc through to the end of the expr.
	(c_parser_unary_expression): Likewise, based on the
	op_loc through to the end of op.
	(c_parser_sizeof_expression) Likewise, based on the start of the
	sizeof token through to either the closing paren or the end of
	expr.
	(c_parser_postfix_expression): Likewise, using the token range,
	or from the open paren through to the close paren for
	parenthesized expressions.
	(c_parser_postfix_expression_after_primary): Likewise, for
	various kinds of expression.
	* c-tree.h (struct c_expr): Add field "src_range".
	(c_expr::get_start): New method.
	(c_expr::get_finish): New method.
	(set_c_expr_source_range): New decls.
	* c-typeck.c (parser_build_unary_op): Call set_c_expr_source_range
	on ret for prefix unary ops.
	(parser_build_binary_op): Likewise, running from the start of
	arg1.value through to the end of arg2.value.

gcc/cp/ChangeLog:
	* error.c (pedwarn_cxx98): Pass line_table to rich_location ctor.

gcc/fortran/ChangeLog:
	* error.c (gfc_warning): Pass line_table to rich_location ctor.
	(gfc_warning_now_at): Likewise.
	(gfc_warning_now): Likewise.
	(gfc_error_now): Likewise.
	(gfc_fatal_error): Likewise.
	(gfc_error): Likewise.
	(gfc_internal_error): Likewise.

gcc/testsuite/ChangeLog:
	* gcc.dg/diagnostic-token-ranges.c: New file.
	* gcc.dg/diagnostic-tree-expr-ranges-2.c: New file.
	* gcc.dg/plugin/diagnostic-test-expressions-1.c: New file.
	* gcc.dg/plugin/diagnostic-test-show-trees-1.c: New file.
	* gcc.dg/plugin/diagnostic_plugin_show_trees.c: New file.
	* gcc.dg/plugin/diagnostic_plugin_test_show_locus.c (get_loc): Add
	line_table param when calling
	linemap_position_for_line_and_column.
	(test_show_locus): Pass line_table to rich_location ctors.
	(plugin_init): Remove setting of global_dc->colorize_source_p.
	* gcc.dg/plugin/diagnostic_plugin_test_tree_expression_range.c:
	New file.
	* gcc.dg/plugin/plugin.exp (plugin_test_list): Add
	diagnostic_plugin_test_tree_expression_range.c,
	diagnostic-test-expressions-1.c, diagnostic_plugin_show_trees.c,
	and diagnostic-test-show-trees-1.c.

libcpp/ChangeLog:
	* errors.c (cpp_diagnostic): Pass pfile->line_table to
	rich_location ctor.
	(cpp_diagnostic_with_line): Likewise.
	* include/cpplib.h (struct cpp_token): Update comment for src_loc
	to indicate that the range of the token is "baked into" the
	source_location.
	* include/line-map.h (source_location): Update the descriptive
	comment to reflect the packing scheme for short ranges, adding
	worked examples of location encoding.
	(struct line_map_ordinary): Drop field "column_bits" in favor
	of field "m_column_and_range_bits"; add field "m_range_bits".
	(ORDINARY_MAP_NUMBER_OF_COLUMN_BITS): Delete.
	(location_adhoc_data): Add source_range field.
	(struct line_maps): Add fields "default_range_bits",
	"num_optimized_ranges" and "num_unoptimized_ranges".
	(get_combined_adhoc_loc): Add source_range param.
	(get_range_from_loc): New declaration.
	(pure_location_p): New prototype.
	(COMBINE_LOCATION_DATA):  Add source_range param.
	(SOURCE_LINE): Update for renaming of column_bits.
	(SOURCE_COLUMN): Likewise.  Shift the column right by the map's
	range_bits.
	(LAST_SOURCE_LINE_LOCATION): Update for renaming of column_bits.
	(linemap_position_for_line_and_column): Add line_maps * params.
	(rich_location::rich_location): Likewise.
	* lex.c (_cpp_lex_direct): Capture the range of the token, baking
	it into token->src_loc via a call to COMBINE_LOCATION_DATA.
	* line-map.c (LINE_MAP_MAX_COLUMN_NUMBER): Reduce from 1U << 17 to
	1U << 12.
	(location_adhoc_data_hash): Add the src_range into
	the hash value.
	(location_adhoc_data_eq): Require equality of the src_range
	values.
	(can_be_stored_compactly_p): New function.
	(get_combined_adhoc_loc): Add src_range param, and store it,
	via a bit-packing scheme for short ranges, otherwise within the
	lookaside table.  Remove the requirement that data is non-NULL.
	(get_range_from_adhoc_loc): New function.
	(get_range_from_loc): New function.
	(pure_location_p): New function.
	(linemap_add): Ensure that start_location has zero for the
	range_bits, unless we're past LINE_MAP_MAX_LOCATION_WITH_COLS.
	Initialize range_bits to zero.  Assert that the start_location
	is "pure".
	(linemap_line_start): Assert that the
	column_and_range_bits >= range_bits.
	Update determinination of whether we need to start a new map
	using the effective column bits, without the range bits.
	Use the set's default_range_bits in new maps, apart from
	those with column_bits == 0, which should also have 0 range_bits.
	Increase the column bits for new maps by the range bits.
	When adding lines to an existing map, use set->highest_line
	directly rather than offsetting highest by SOURCE_COLUMN.
	Add assertions to sanity-check the return value.
	(linemap_position_for_column): Offset to_column by range_bits.
	Update set->highest_location if necessary.
	(linemap_position_for_line_and_column): Add line_maps * param.
	Update the calculation to offset the column by range_bits, and
	conditionalize it on being <= LINE_MAP_MAX_LOCATION_WITH_COLS.
	Bound it by LINEMAPS_MACRO_LOWEST_LOCATION.  Update
	set->highest_location if necessary.
	(linemap_position_for_loc_and_offset): Handle ad-hoc locations;
	pass "set" to linemap_position_for_line_and_column.
	(linemap_macro_map_loc_unwind_toward_spelling): Add line_maps
	param.  Handle ad-hoc locations.
	(linemap_location_in_system_header_p): Pass on "set" to call to
	linemap_macro_map_loc_unwind_toward_spelling.
	(linemap_macro_loc_to_spelling_point): Retain ad-hoc locations.
	Pass on "set" to call to
	linemap_macro_map_loc_unwind_toward_spelling.
	(linemap_resolve_location): Retain ad-hoc locations.  Pass on
	"set" to call to linemap_macro_map_loc_unwind_toward_spelling.
	(linemap_unwind_toward_expansion):  Pass on "set" to call to
	linemap_macro_map_loc_unwind_toward_spelling.
	(linemap_expand_location): Extract the data pointer before
	extracting the location.
	(rich_location::rich_location): Add line_maps param; use it to
	extract the range from the source_location.
	* location-example.txt: Regenerate, showing new representation.
---
 gcc/Makefile.in                                    |   1 +
 gcc/ada/gcc-interface/trans.c                      |   3 +-
 gcc/c-family/c-common.c                            |  10 +-
 gcc/c-family/c-opts.c                              |   2 +
 gcc/c/c-decl.c                                     |   2 +-
 gcc/c/c-errors.c                                   |   4 +-
 gcc/c/c-parser.c                                   |  92 ++++-
 gcc/c/c-tree.h                                     |  19 +
 gcc/c/c-typeck.c                                   |  10 +
 gcc/cp/error.c                                     |   2 +-
 gcc/diagnostic.c                                   |  34 +-
 gcc/fortran/error.c                                |  14 +-
 gcc/gcc-rich-location.c                            |  86 +++++
 gcc/gcc-rich-location.h                            |  47 +++
 gcc/genmatch.c                                     |   8 +-
 gcc/gimple.h                                       |   6 +-
 gcc/input.c                                        |  28 +-
 gcc/print-tree.c                                   |  21 +
 gcc/rtl-error.c                                    |   2 +-
 gcc/testsuite/gcc.dg/diagnostic-token-ranges.c     | 120 ++++++
 .../gcc.dg/diagnostic-tree-expr-ranges-2.c         |  23 ++
 .../gcc.dg/plugin/diagnostic-test-expressions-1.c  | 422 +++++++++++++++++++++
 .../gcc.dg/plugin/diagnostic-test-show-trees-1.c   |  65 ++++
 .../gcc.dg/plugin/diagnostic_plugin_show_trees.c   | 174 +++++++++
 .../plugin/diagnostic_plugin_test_show_locus.c     |  24 +-
 .../diagnostic_plugin_test_tree_expression_range.c |  98 +++++
 gcc/testsuite/gcc.dg/plugin/plugin.exp             |   4 +
 gcc/toplev.c                                       |   1 +
 gcc/tree-cfg.c                                     |   9 +-
 gcc/tree-inline.c                                  |   5 +-
 gcc/tree.c                                         |  60 ++-
 gcc/tree.h                                         |  33 ++
 libcpp/errors.c                                    |   4 +-
 libcpp/include/cpplib.h                            |   3 +-
 libcpp/include/line-map.h                          | 219 +++++++++--
 libcpp/lex.c                                       |  13 +
 libcpp/line-map.c                                  | 274 +++++++++++--
 libcpp/location-example.txt                        | 188 ++++-----
 38 files changed, 1888 insertions(+), 242 deletions(-)
 create mode 100644 gcc/gcc-rich-location.c
 create mode 100644 gcc/gcc-rich-location.h
 create mode 100644 gcc/testsuite/gcc.dg/diagnostic-token-ranges.c
 create mode 100644 gcc/testsuite/gcc.dg/diagnostic-tree-expr-ranges-2.c
 create mode 100644 gcc/testsuite/gcc.dg/plugin/diagnostic-test-expressions-1.c
 create mode 100644 gcc/testsuite/gcc.dg/plugin/diagnostic-test-show-trees-1.c
 create mode 100644 gcc/testsuite/gcc.dg/plugin/diagnostic_plugin_show_trees.c
 create mode 100644 gcc/testsuite/gcc.dg/plugin/diagnostic_plugin_test_tree_expression_range.c

diff --git a/gcc/Makefile.in b/gcc/Makefile.in
index 34d2356..bd6f484 100644
--- a/gcc/Makefile.in
+++ b/gcc/Makefile.in
@@ -1263,6 +1263,7 @@ OBJS = \
 	fold-const-call.o \
 	function.o \
 	fwprop.o \
+	gcc-rich-location.o \
 	gcse.o \
 	gcse-common.o \
 	ggc-common.o \
diff --git a/gcc/ada/gcc-interface/trans.c b/gcc/ada/gcc-interface/trans.c
index ca66a03..eeb2aac 100644
--- a/gcc/ada/gcc-interface/trans.c
+++ b/gcc/ada/gcc-interface/trans.c
@@ -9650,7 +9650,8 @@ Sloc_to_locus (Source_Ptr Sloc, location_t *locus, bool clear_column)
     line = 1;
 
   /* Translate the location.  */
-  *locus = linemap_position_for_line_and_column (map, line, column);
+  *locus = linemap_position_for_line_and_column (line_table, map,
+						 line, column);
 
   return true;
 }
diff --git a/gcc/c-family/c-common.c b/gcc/c-family/c-common.c
index 6e2ce0a..89e978d 100644
--- a/gcc/c-family/c-common.c
+++ b/gcc/c-family/c-common.c
@@ -1187,6 +1187,7 @@ c_fully_fold_internal (tree expr, bool in_init, bool *maybe_const_operands,
   bool op0_const_self = true, op1_const_self = true, op2_const_self = true;
   bool nowarning = TREE_NO_WARNING (expr);
   bool unused_p;
+  source_range old_range;
 
   /* This function is not relevant to C++ because C++ folds while
      parsing, and may need changes to be correct for C++ when C++
@@ -1202,6 +1203,9 @@ c_fully_fold_internal (tree expr, bool in_init, bool *maybe_const_operands,
       || code == SAVE_EXPR)
     return expr;
 
+  if (IS_EXPR_CODE_CLASS (kind))
+    old_range = EXPR_LOCATION_RANGE (expr);
+
   /* Operands of variable-length expressions (function calls) have
      already been folded, as have __builtin_* function calls, and such
      expressions cannot occur in constant expressions.  */
@@ -1626,7 +1630,11 @@ c_fully_fold_internal (tree expr, bool in_init, bool *maybe_const_operands,
       TREE_NO_WARNING (ret) = 1;
     }
   if (ret != expr)
-    protected_set_expr_location (ret, loc);
+    {
+      protected_set_expr_location (ret, loc);
+      if (IS_EXPR_CODE_CLASS (kind))
+	set_source_range (ret, old_range.m_start, old_range.m_finish);
+    }
   return ret;
 }
 
diff --git a/gcc/c-family/c-opts.c b/gcc/c-family/c-opts.c
index 4da6f31..9ae181f 100644
--- a/gcc/c-family/c-opts.c
+++ b/gcc/c-family/c-opts.c
@@ -245,6 +245,8 @@ c_common_init_options (unsigned int decoded_options_count,
 	    break;
 	  }
     }
+
+  global_dc->colorize_source_p = true;
 }
 
 /* Handle switch SCODE with argument ARG.  VALUE is true, unless no-
diff --git a/gcc/c/c-decl.c b/gcc/c/c-decl.c
index a636474..9a222d8 100644
--- a/gcc/c/c-decl.c
+++ b/gcc/c/c-decl.c
@@ -5278,7 +5278,7 @@ warn_defaults_to (location_t location, int opt, const char *gmsgid, ...)
 {
   diagnostic_info diagnostic;
   va_list ap;
-  rich_location richloc (location);
+  rich_location richloc (line_table, location);
 
   va_start (ap, gmsgid);
   diagnostic_set_info (&diagnostic, gmsgid, &ap, &richloc,
diff --git a/gcc/c/c-errors.c b/gcc/c/c-errors.c
index ef0f9a2..ee9c2b5 100644
--- a/gcc/c/c-errors.c
+++ b/gcc/c/c-errors.c
@@ -37,7 +37,7 @@ pedwarn_c99 (location_t location, int opt, const char *gmsgid, ...)
   diagnostic_info diagnostic;
   va_list ap;
   bool warned = false;
-  rich_location richloc (location);
+  rich_location richloc (line_table, location);
 
   va_start (ap, gmsgid);
   /* If desired, issue the C99/C11 compat warning, which is more specific
@@ -76,7 +76,7 @@ pedwarn_c90 (location_t location, int opt, const char *gmsgid, ...)
 {
   diagnostic_info diagnostic;
   va_list ap;
-  rich_location richloc (location);
+  rich_location richloc (line_table, location);
 
   va_start (ap, gmsgid);
   /* Warnings such as -Wvla are the most specific ones.  */
diff --git a/gcc/c/c-parser.c b/gcc/c/c-parser.c
index 2484b92..24a7010 100644
--- a/gcc/c/c-parser.c
+++ b/gcc/c/c-parser.c
@@ -59,6 +59,23 @@ along with GCC; see the file COPYING3.  If not see
 #include "gimple-expr.h"
 #include "context.h"
 
+void
+set_c_expr_source_range (c_expr *expr,
+			 location_t start, location_t finish)
+{
+  expr->src_range.m_start = start;
+  expr->src_range.m_finish = finish;
+  set_source_range (expr->value, start, finish);
+}
+
+void
+set_c_expr_source_range (c_expr *expr,
+			 source_range src_range)
+{
+  expr->src_range = src_range;
+  set_source_range (expr->value, src_range);
+}
+
 
 /* Initialization routine for this file.  */
 
@@ -164,6 +181,16 @@ struct GTY (()) c_token {
   location_t location;
   /* The value associated with this token, if any.  */
   tree value;
+
+  source_range get_range () const
+  {
+    return get_range_from_loc (line_table, location);
+  }
+
+  location_t get_finish () const
+  {
+    return get_range ().m_finish;
+  }
 };
 
 /* A parser structure recording information about the state and
@@ -6101,6 +6128,7 @@ c_parser_expr_no_commas (c_parser *parser, struct c_expr *after,
   ret.value = build_modify_expr (op_location, lhs.value, lhs.original_type,
 				 code, exp_location, rhs.value,
 				 rhs.original_type);
+  set_c_expr_source_range (&ret, lhs.get_start (), rhs.get_finish ());
   if (code == NOP_EXPR)
     ret.original_code = MODIFY_EXPR;
   else
@@ -6131,7 +6159,7 @@ c_parser_conditional_expression (c_parser *parser, struct c_expr *after,
 				 tree omp_atomic_lhs)
 {
   struct c_expr cond, exp1, exp2, ret;
-  location_t cond_loc, colon_loc, middle_loc;
+  location_t start, cond_loc, colon_loc, middle_loc;
 
   gcc_assert (!after || c_dialect_objc ());
 
@@ -6139,6 +6167,10 @@ c_parser_conditional_expression (c_parser *parser, struct c_expr *after,
 
   if (c_parser_next_token_is_not (parser, CPP_QUERY))
     return cond;
+  if (cond.value != error_mark_node)
+    start = cond.get_start ();
+  else
+    start = UNKNOWN_LOCATION;
   cond_loc = c_parser_peek_token (parser)->location;
   cond = convert_lvalue_to_rvalue (cond_loc, cond, true, true);
   c_parser_consume_token (parser);
@@ -6214,6 +6246,7 @@ c_parser_conditional_expression (c_parser *parser, struct c_expr *after,
 			   ? t1
 			   : NULL);
     }
+  set_c_expr_source_range (&ret, start, exp2.get_finish ());
   return ret;
 }
 
@@ -6366,6 +6399,7 @@ c_parser_binary_expression (c_parser *parser, struct c_expr *after,
     {
       enum c_parser_prec oprec;
       enum tree_code ocode;
+      source_range src_range;
       if (parser->error)
 	goto out;
       switch (c_parser_peek_token (parser)->type)
@@ -6454,6 +6488,7 @@ c_parser_binary_expression (c_parser *parser, struct c_expr *after,
       switch (ocode)
 	{
 	case TRUTH_ANDIF_EXPR:
+	  src_range = stack[sp].expr.src_range;
 	  stack[sp].expr
 	    = convert_lvalue_to_rvalue (stack[sp].loc,
 					stack[sp].expr, true, true);
@@ -6461,8 +6496,10 @@ c_parser_binary_expression (c_parser *parser, struct c_expr *after,
 	    (stack[sp].loc, default_conversion (stack[sp].expr.value));
 	  c_inhibit_evaluation_warnings += (stack[sp].expr.value
 					    == truthvalue_false_node);
+	  set_c_expr_source_range (&stack[sp].expr, src_range);
 	  break;
 	case TRUTH_ORIF_EXPR:
+	  src_range = stack[sp].expr.src_range;
 	  stack[sp].expr
 	    = convert_lvalue_to_rvalue (stack[sp].loc,
 					stack[sp].expr, true, true);
@@ -6470,6 +6507,7 @@ c_parser_binary_expression (c_parser *parser, struct c_expr *after,
 	    (stack[sp].loc, default_conversion (stack[sp].expr.value));
 	  c_inhibit_evaluation_warnings += (stack[sp].expr.value
 					    == truthvalue_true_node);
+	  set_c_expr_source_range (&stack[sp].expr, src_range);
 	  break;
 	default:
 	  break;
@@ -6538,6 +6576,8 @@ c_parser_cast_expression (c_parser *parser, struct c_expr *after)
 	expr = convert_lvalue_to_rvalue (expr_loc, expr, true, true);
       }
       ret.value = c_cast_expr (cast_loc, type_name, expr.value);
+      if (ret.value && expr.value)
+	set_c_expr_source_range (&ret, cast_loc, expr.get_finish ());
       ret.original_code = ERROR_MARK;
       ret.original_type = NULL;
       return ret;
@@ -6587,6 +6627,7 @@ c_parser_unary_expression (c_parser *parser)
   struct c_expr ret, op;
   location_t op_loc = c_parser_peek_token (parser)->location;
   location_t exp_loc;
+  location_t finish;
   ret.original_code = ERROR_MARK;
   ret.original_type = NULL;
   switch (c_parser_peek_token (parser)->type)
@@ -6626,8 +6667,10 @@ c_parser_unary_expression (c_parser *parser)
       c_parser_consume_token (parser);
       exp_loc = c_parser_peek_token (parser)->location;
       op = c_parser_cast_expression (parser, NULL);
+      finish = op.get_finish ();
       op = convert_lvalue_to_rvalue (exp_loc, op, true, true);
       ret.value = build_indirect_ref (op_loc, op.value, RO_UNARY_STAR);
+      set_c_expr_source_range (&ret, op_loc, finish);
       return ret;
     case CPP_PLUS:
       if (!c_dialect_objc () && !in_system_header_at (input_location))
@@ -6715,8 +6758,15 @@ static struct c_expr
 c_parser_sizeof_expression (c_parser *parser)
 {
   struct c_expr expr;
+  struct c_expr result;
   location_t expr_loc;
   gcc_assert (c_parser_next_token_is_keyword (parser, RID_SIZEOF));
+
+  location_t start;
+  location_t finish = UNKNOWN_LOCATION;
+
+  start = c_parser_peek_token (parser)->location;
+
   c_parser_consume_token (parser);
   c_inhibit_evaluation_warnings++;
   in_sizeof++;
@@ -6730,6 +6780,7 @@ c_parser_sizeof_expression (c_parser *parser)
       expr_loc = c_parser_peek_token (parser)->location;
       type_name = c_parser_type_name (parser);
       c_parser_skip_until_found (parser, CPP_CLOSE_PAREN, "expected %<)%>");
+      finish = parser->tokens_buf[0].location;
       if (type_name == NULL)
 	{
 	  struct c_expr ret;
@@ -6745,17 +6796,19 @@ c_parser_sizeof_expression (c_parser *parser)
 	  expr = c_parser_postfix_expression_after_paren_type (parser,
 							       type_name,
 							       expr_loc);
+	  finish = expr.get_finish ();
 	  goto sizeof_expr;
 	}
       /* sizeof ( type-name ).  */
       c_inhibit_evaluation_warnings--;
       in_sizeof--;
-      return c_expr_sizeof_type (expr_loc, type_name);
+      result = c_expr_sizeof_type (expr_loc, type_name);
     }
   else
     {
       expr_loc = c_parser_peek_token (parser)->location;
       expr = c_parser_unary_expression (parser);
+      finish = expr.get_finish ();
     sizeof_expr:
       c_inhibit_evaluation_warnings--;
       in_sizeof--;
@@ -6763,8 +6816,11 @@ c_parser_sizeof_expression (c_parser *parser)
       if (TREE_CODE (expr.value) == COMPONENT_REF
 	  && DECL_C_BIT_FIELD (TREE_OPERAND (expr.value, 1)))
 	error_at (expr_loc, "%<sizeof%> applied to a bit-field");
-      return c_expr_sizeof_expr (expr_loc, expr);
+      result = c_expr_sizeof_expr (expr_loc, expr);
     }
+  if (finish != UNKNOWN_LOCATION)
+    set_c_expr_source_range (&result, start, finish);
+  return result;
 }
 
 /* Parse an alignof expression.  */
@@ -7184,12 +7240,14 @@ c_parser_postfix_expression (c_parser *parser)
   struct c_expr expr, e1;
   struct c_type_name *t1, *t2;
   location_t loc = c_parser_peek_token (parser)->location;;
+  source_range tok_range = c_parser_peek_token (parser)->get_range ();
   expr.original_code = ERROR_MARK;
   expr.original_type = NULL;
   switch (c_parser_peek_token (parser)->type)
     {
     case CPP_NUMBER:
       expr.value = c_parser_peek_token (parser)->value;
+      set_c_expr_source_range (&expr, tok_range);
       loc = c_parser_peek_token (parser)->location;
       c_parser_consume_token (parser);
       if (TREE_CODE (expr.value) == FIXED_CST
@@ -7204,6 +7262,7 @@ c_parser_postfix_expression (c_parser *parser)
     case CPP_CHAR32:
     case CPP_WCHAR:
       expr.value = c_parser_peek_token (parser)->value;
+      set_c_expr_source_range (&expr, tok_range);
       c_parser_consume_token (parser);
       break;
     case CPP_STRING:
@@ -7212,6 +7271,7 @@ c_parser_postfix_expression (c_parser *parser)
     case CPP_WSTRING:
     case CPP_UTF8STRING:
       expr.value = c_parser_peek_token (parser)->value;
+      set_c_expr_source_range (&expr, tok_range);
       expr.original_code = STRING_CST;
       c_parser_consume_token (parser);
       break;
@@ -7219,6 +7279,7 @@ c_parser_postfix_expression (c_parser *parser)
       gcc_assert (c_dialect_objc ());
       expr.value
 	= objc_build_string_object (c_parser_peek_token (parser)->value);
+      set_c_expr_source_range (&expr, tok_range);
       c_parser_consume_token (parser);
       break;
     case CPP_NAME:
@@ -7232,6 +7293,7 @@ c_parser_postfix_expression (c_parser *parser)
 					     (c_parser_peek_token (parser)->type
 					      == CPP_OPEN_PAREN),
 					     &expr.original_type);
+	    set_c_expr_source_range (&expr, tok_range);
 	    break;
 	  }
 	case C_ID_CLASSNAME:
@@ -7320,6 +7382,7 @@ c_parser_postfix_expression (c_parser *parser)
       else
 	{
 	  /* A parenthesized expression.  */
+	  location_t loc_open_paren = c_parser_peek_token (parser)->location;
 	  c_parser_consume_token (parser);
 	  expr = c_parser_expression (parser);
 	  if (TREE_CODE (expr.value) == MODIFY_EXPR)
@@ -7327,6 +7390,8 @@ c_parser_postfix_expression (c_parser *parser)
 	  if (expr.original_code != C_MAYBE_CONST_EXPR)
 	    expr.original_code = ERROR_MARK;
 	  /* Don't change EXPR.ORIGINAL_TYPE.  */
+	  location_t loc_close_paren = c_parser_peek_token (parser)->location;
+	  set_c_expr_source_range (&expr, loc_open_paren, loc_close_paren);
 	  c_parser_skip_until_found (parser, CPP_CLOSE_PAREN,
 				     "expected %<)%>");
 	}
@@ -7917,6 +7982,8 @@ c_parser_postfix_expression_after_primary (c_parser *parser,
   vec<tree, va_gc> *exprlist;
   vec<tree, va_gc> *origtypes = NULL;
   vec<location_t> arg_loc = vNULL;
+  location_t start;
+  location_t finish;
 
   while (true)
     {
@@ -7953,7 +8020,10 @@ c_parser_postfix_expression_after_primary (c_parser *parser,
 		{
 		  c_parser_skip_until_found (parser, CPP_CLOSE_SQUARE,
 					     "expected %<]%>");
+		  start = expr.get_start ();
+		  finish = parser->tokens_buf[0].location;
 		  expr.value = build_array_ref (op_loc, expr.value, idx);
+		  set_c_expr_source_range (&expr, start, finish);
 		}
 	    }
 	  expr.original_code = ERROR_MARK;
@@ -7996,9 +8066,13 @@ c_parser_postfix_expression_after_primary (c_parser *parser,
 			"%<memset%> used with constant zero length parameter; "
 			"this could be due to transposed parameters");
 
+	  start = expr.get_start ();
+	  finish = parser->tokens_buf[0].get_finish ();
 	  expr.value
 	    = c_build_function_call_vec (expr_loc, arg_loc, expr.value,
 					 exprlist, origtypes);
+	  set_c_expr_source_range (&expr, start, finish);
+
 	  expr.original_code = ERROR_MARK;
 	  if (TREE_CODE (expr.value) == INTEGER_CST
 	      && TREE_CODE (orig_expr.value) == FUNCTION_DECL
@@ -8027,8 +8101,11 @@ c_parser_postfix_expression_after_primary (c_parser *parser,
               expr.original_type = NULL;
 	      return expr;
 	    }
+	  start = expr.get_start ();
+	  finish = c_parser_peek_token (parser)->get_finish ();
 	  c_parser_consume_token (parser);
 	  expr.value = build_component_ref (op_loc, expr.value, ident);
+	  set_c_expr_source_range (&expr, start, finish);
 	  expr.original_code = ERROR_MARK;
 	  if (TREE_CODE (expr.value) != COMPONENT_REF)
 	    expr.original_type = NULL;
@@ -8056,12 +8133,15 @@ c_parser_postfix_expression_after_primary (c_parser *parser,
 	      expr.original_type = NULL;
 	      return expr;
 	    }
+	  start = expr.get_start ();
+	  finish = c_parser_peek_token (parser)->get_finish ();
 	  c_parser_consume_token (parser);
 	  expr.value = build_component_ref (op_loc,
 					    build_indirect_ref (op_loc,
 								expr.value,
 								RO_ARROW),
 					    ident);
+	  set_c_expr_source_range (&expr, start, finish);
 	  expr.original_code = ERROR_MARK;
 	  if (TREE_CODE (expr.value) != COMPONENT_REF)
 	    expr.original_type = NULL;
@@ -8077,6 +8157,8 @@ c_parser_postfix_expression_after_primary (c_parser *parser,
 	  break;
 	case CPP_PLUS_PLUS:
 	  /* Postincrement.  */
+	  start = expr.get_start ();
+	  finish = c_parser_peek_token (parser)->get_finish ();
 	  c_parser_consume_token (parser);
 	  /* If the expressions have array notations, we expand them.  */
 	  if (flag_cilkplus
@@ -8088,11 +8170,14 @@ c_parser_postfix_expression_after_primary (c_parser *parser,
 	      expr.value = build_unary_op (op_loc,
 					   POSTINCREMENT_EXPR, expr.value, 0);
 	    }
+	  set_c_expr_source_range (&expr, start, finish);
 	  expr.original_code = ERROR_MARK;
 	  expr.original_type = NULL;
 	  break;
 	case CPP_MINUS_MINUS:
 	  /* Postdecrement.  */
+	  start = expr.get_start ();
+	  finish = c_parser_peek_token (parser)->get_finish ();
 	  c_parser_consume_token (parser);
 	  /* If the expressions have array notations, we expand them.  */
 	  if (flag_cilkplus
@@ -8104,6 +8189,7 @@ c_parser_postfix_expression_after_primary (c_parser *parser,
 	      expr.value = build_unary_op (op_loc,
 					   POSTDECREMENT_EXPR, expr.value, 0);
 	    }
+	  set_c_expr_source_range (&expr, start, finish);
 	  expr.original_code = ERROR_MARK;
 	  expr.original_type = NULL;
 	  break;
diff --git a/gcc/c/c-tree.h b/gcc/c/c-tree.h
index 04991f7..6bc216a 100644
--- a/gcc/c/c-tree.h
+++ b/gcc/c/c-tree.h
@@ -132,6 +132,17 @@ struct c_expr
      The type of an enum constant is a plain integer type, but this
      field will be the enum type.  */
   tree original_type;
+
+  /* The source range of this expression.  This is redundant
+     for node values that have locations, but not all node kinds
+     have locations (e.g. constants, and references to params, locals,
+     etc), so we stash a copy here.  */
+  source_range src_range;
+
+  /* Access to the first and last locations within the source spelling
+     of this expression.  */
+  location_t get_start () const { return src_range.m_start; }
+  location_t get_finish () const { return src_range.m_finish; }
 };
 
 /* Type alias for struct c_expr. This allows to use the structure
@@ -708,4 +719,12 @@ extern void pedwarn_c90 (location_t, int opt, const char *, ...)
 extern bool pedwarn_c99 (location_t, int opt, const char *, ...)
     ATTRIBUTE_GCC_DIAG(3,4);
 
+extern void
+set_c_expr_source_range (c_expr *expr,
+			 location_t start, location_t finish);
+
+extern void
+set_c_expr_source_range (c_expr *expr,
+			 source_range src_range);
+
 #endif /* ! GCC_C_TREE_H */
diff --git a/gcc/c/c-typeck.c b/gcc/c/c-typeck.c
index 4335a87..0c3fa19 100644
--- a/gcc/c/c-typeck.c
+++ b/gcc/c/c-typeck.c
@@ -3388,6 +3388,12 @@ parser_build_unary_op (location_t loc, enum tree_code code, struct c_expr arg)
     overflow_warning (loc, result.value);
     }
 
+  /* We are typically called when parsing a prefix token at LOC acting on
+     ARG.  Reflect this by updating the source range of the result to
+     start at LOC and end at the end of ARG.  */
+  set_c_expr_source_range (&result,
+			   loc, arg.get_finish ());
+
   return result;
 }
 
@@ -3425,6 +3431,10 @@ parser_build_binary_op (location_t location, enum tree_code code,
   if (location != UNKNOWN_LOCATION)
     protected_set_expr_location (result.value, location);
 
+  set_c_expr_source_range (&result,
+			   arg1.get_start (),
+			   arg2.get_finish ());
+
   /* Check for cases such as x+y<<z which users are likely
      to misinterpret.  */
   if (warn_parentheses)
diff --git a/gcc/cp/error.c b/gcc/cp/error.c
index 361e41a..38548c7 100644
--- a/gcc/cp/error.c
+++ b/gcc/cp/error.c
@@ -3673,7 +3673,7 @@ pedwarn_cxx98 (location_t location, int opt, const char *gmsgid, ...)
   diagnostic_info diagnostic;
   va_list ap;
   bool ret;
-  rich_location richloc (location);
+  rich_location richloc (line_table, location);
 
   va_start (ap, gmsgid);
   diagnostic_set_info (&diagnostic, gmsgid, &ap, &richloc,
diff --git a/gcc/diagnostic.c b/gcc/diagnostic.c
index ee034e7..b4d3a7d 100644
--- a/gcc/diagnostic.c
+++ b/gcc/diagnostic.c
@@ -867,7 +867,7 @@ diagnostic_append_note (diagnostic_context *context,
   diagnostic_info diagnostic;
   va_list ap;
   const char *saved_prefix;
-  rich_location richloc (location);
+  rich_location richloc (line_table, location);
 
   va_start (ap, gmsgid);
   diagnostic_set_info (&diagnostic, gmsgid, &ap, &richloc, DK_NOTE);
@@ -925,7 +925,7 @@ emit_diagnostic (diagnostic_t kind, location_t location, int opt,
   diagnostic_info diagnostic;
   va_list ap;
   bool ret;
-  rich_location richloc (location);
+  rich_location richloc (line_table, location);
 
   va_start (ap, gmsgid);
   if (kind == DK_PERMERROR)
@@ -952,7 +952,7 @@ inform (location_t location, const char *gmsgid, ...)
 {
   diagnostic_info diagnostic;
   va_list ap;
-  rich_location richloc (location);
+  rich_location richloc (line_table, location);
 
   va_start (ap, gmsgid);
   diagnostic_set_info (&diagnostic, gmsgid, &ap, &richloc, DK_NOTE);
@@ -981,7 +981,7 @@ inform_n (location_t location, int n, const char *singular_gmsgid,
 {
   diagnostic_info diagnostic;
   va_list ap;
-  rich_location richloc (location);
+  rich_location richloc (line_table, location);
 
   va_start (ap, plural_gmsgid);
   diagnostic_set_info_translated (&diagnostic,
@@ -1000,7 +1000,7 @@ warning (int opt, const char *gmsgid, ...)
   diagnostic_info diagnostic;
   va_list ap;
   bool ret;
-  rich_location richloc (input_location);
+  rich_location richloc (line_table, input_location);
 
   va_start (ap, gmsgid);
   diagnostic_set_info (&diagnostic, gmsgid, &ap, &richloc, DK_WARNING);
@@ -1021,7 +1021,7 @@ warning_at (location_t location, int opt, const char *gmsgid, ...)
   diagnostic_info diagnostic;
   va_list ap;
   bool ret;
-  rich_location richloc (location);
+  rich_location richloc (line_table, location);
 
   va_start (ap, gmsgid);
   diagnostic_set_info (&diagnostic, gmsgid, &ap, &richloc, DK_WARNING);
@@ -1059,7 +1059,7 @@ warning_n (location_t location, int opt, int n, const char *singular_gmsgid,
   diagnostic_info diagnostic;
   va_list ap;
   bool ret;
-  rich_location richloc (location);
+  rich_location richloc (line_table, location);
 
   va_start (ap, plural_gmsgid);
   diagnostic_set_info_translated (&diagnostic,
@@ -1091,7 +1091,7 @@ pedwarn (location_t location, int opt, const char *gmsgid, ...)
   diagnostic_info diagnostic;
   va_list ap;
   bool ret;
-  rich_location richloc (location);
+  rich_location richloc (line_table, location);
 
   va_start (ap, gmsgid);
   diagnostic_set_info (&diagnostic, gmsgid, &ap, &richloc,  DK_PEDWARN);
@@ -1114,7 +1114,7 @@ permerror (location_t location, const char *gmsgid, ...)
   diagnostic_info diagnostic;
   va_list ap;
   bool ret;
-  rich_location richloc (location);
+  rich_location richloc (line_table, location);
 
   va_start (ap, gmsgid);
   diagnostic_set_info (&diagnostic, gmsgid, &ap, &richloc,
@@ -1150,7 +1150,7 @@ error (const char *gmsgid, ...)
 {
   diagnostic_info diagnostic;
   va_list ap;
-  rich_location richloc (input_location);
+  rich_location richloc (line_table, input_location);
 
   va_start (ap, gmsgid);
   diagnostic_set_info (&diagnostic, gmsgid, &ap, &richloc, DK_ERROR);
@@ -1166,7 +1166,7 @@ error_n (location_t location, int n, const char *singular_gmsgid,
 {
   diagnostic_info diagnostic;
   va_list ap;
-  rich_location richloc (location);
+  rich_location richloc (line_table, location);
 
   va_start (ap, plural_gmsgid);
   diagnostic_set_info_translated (&diagnostic,
@@ -1182,7 +1182,7 @@ error_at (location_t loc, const char *gmsgid, ...)
 {
   diagnostic_info diagnostic;
   va_list ap;
-  rich_location richloc (loc);
+  rich_location richloc (line_table, loc);
 
   va_start (ap, gmsgid);
   diagnostic_set_info (&diagnostic, gmsgid, &ap, &richloc, DK_ERROR);
@@ -1213,7 +1213,7 @@ sorry (const char *gmsgid, ...)
 {
   diagnostic_info diagnostic;
   va_list ap;
-  rich_location richloc (input_location);
+  rich_location richloc (line_table, input_location);
 
   va_start (ap, gmsgid);
   diagnostic_set_info (&diagnostic, gmsgid, &ap, &richloc, DK_SORRY);
@@ -1237,7 +1237,7 @@ fatal_error (location_t loc, const char *gmsgid, ...)
 {
   diagnostic_info diagnostic;
   va_list ap;
-  rich_location richloc (loc);
+  rich_location richloc (line_table, loc);
 
   va_start (ap, gmsgid);
   diagnostic_set_info (&diagnostic, gmsgid, &ap, &richloc, DK_FATAL);
@@ -1256,7 +1256,7 @@ internal_error (const char *gmsgid, ...)
 {
   diagnostic_info diagnostic;
   va_list ap;
-  rich_location richloc (input_location);
+  rich_location richloc (line_table, input_location);
 
   va_start (ap, gmsgid);
   diagnostic_set_info (&diagnostic, gmsgid, &ap, &richloc, DK_ICE);
@@ -1274,7 +1274,7 @@ internal_error_no_backtrace (const char *gmsgid, ...)
 {
   diagnostic_info diagnostic;
   va_list ap;
-  rich_location richloc (input_location);
+  rich_location richloc (line_table, input_location);
 
   va_start (ap, gmsgid);
   diagnostic_set_info (&diagnostic, gmsgid, &ap, &richloc, DK_ICE_NOBT);
@@ -1351,7 +1351,7 @@ real_abort (void)
 DEBUG_FUNCTION void
 source_range::debug (const char *msg) const
 {
-  rich_location richloc (m_start);
+  rich_location richloc (line_table, m_start);
   richloc.add_range (m_start, m_finish, false);
   inform_at_rich_loc (&richloc, "%s", msg);
 }
diff --git a/gcc/fortran/error.c b/gcc/fortran/error.c
index 4b3d31c..b4f7020 100644
--- a/gcc/fortran/error.c
+++ b/gcc/fortran/error.c
@@ -773,7 +773,7 @@ gfc_warning (int opt, const char *gmsgid, va_list ap)
   va_copy (argp, ap);
 
   diagnostic_info diagnostic;
-  rich_location rich_loc (UNKNOWN_LOCATION);
+  rich_location rich_loc (line_table, UNKNOWN_LOCATION);
   bool fatal_errors = global_dc->fatal_errors;
   pretty_printer *pp = global_dc->printer;
   output_buffer *tmp_buffer = pp->buffer;
@@ -1120,7 +1120,7 @@ gfc_warning_now_at (location_t loc, int opt, const char *gmsgid, ...)
 {
   va_list argp;
   diagnostic_info diagnostic;
-  rich_location rich_loc (loc);
+  rich_location rich_loc (line_table, loc);
   bool ret;
 
   va_start (argp, gmsgid);
@@ -1138,7 +1138,7 @@ gfc_warning_now (int opt, const char *gmsgid, ...)
 {
   va_list argp;
   diagnostic_info diagnostic;
-  rich_location rich_loc (UNKNOWN_LOCATION);
+  rich_location rich_loc (line_table, UNKNOWN_LOCATION);
   bool ret;
 
   va_start (argp, gmsgid);
@@ -1158,7 +1158,7 @@ gfc_error_now (const char *gmsgid, ...)
 {
   va_list argp;
   diagnostic_info diagnostic;
-  rich_location rich_loc (UNKNOWN_LOCATION);
+  rich_location rich_loc (line_table, UNKNOWN_LOCATION);
 
   error_buffer.flag = true;
 
@@ -1176,7 +1176,7 @@ gfc_fatal_error (const char *gmsgid, ...)
 {
   va_list argp;
   diagnostic_info diagnostic;
-  rich_location rich_loc (UNKNOWN_LOCATION);
+  rich_location rich_loc (line_table, UNKNOWN_LOCATION);
 
   va_start (argp, gmsgid);
   diagnostic_set_info (&diagnostic, gmsgid, &argp, &rich_loc, DK_FATAL);
@@ -1242,7 +1242,7 @@ gfc_error (const char *gmsgid, va_list ap)
     }
 
   diagnostic_info diagnostic;
-  rich_location richloc (UNKNOWN_LOCATION);
+  rich_location richloc (line_table, UNKNOWN_LOCATION);
   bool fatal_errors = global_dc->fatal_errors;
   pretty_printer *pp = global_dc->printer;
   output_buffer *tmp_buffer = pp->buffer;
@@ -1288,7 +1288,7 @@ gfc_internal_error (const char *gmsgid, ...)
 {
   va_list argp;
   diagnostic_info diagnostic;
-  rich_location rich_loc (UNKNOWN_LOCATION);
+  rich_location rich_loc (line_table, UNKNOWN_LOCATION);
 
   va_start (argp, gmsgid);
   diagnostic_set_info (&diagnostic, gmsgid, &argp, &rich_loc, DK_ICE);
diff --git a/gcc/gcc-rich-location.c b/gcc/gcc-rich-location.c
new file mode 100644
index 0000000..b0ec47b
--- /dev/null
+++ b/gcc/gcc-rich-location.c
@@ -0,0 +1,86 @@
+/* Implementation of gcc_rich_location class
+   Copyright (C) 2014-2015 Free Software Foundation, Inc.
+
+This file is part of GCC.
+
+GCC is free software; you can redistribute it and/or modify it under
+the terms of the GNU General Public License as published by the Free
+Software Foundation; either version 3, or (at your option) any later
+version.
+
+GCC is distributed in the hope that it will be useful, but WITHOUT ANY
+WARRANTY; without even the implied warranty of MERCHANTABILITY or
+FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License
+for more details.
+
+You should have received a copy of the GNU General Public License
+along with GCC; see the file COPYING3.  If not see
+<http://www.gnu.org/licenses/>.  */
+
+#include "config.h"
+#include "system.h"
+#include "coretypes.h"
+#include "tm.h"
+#include "rtl.h"
+#include "hash-set.h"
+#include "machmode.h"
+#include "vec.h"
+#include "double-int.h"
+#include "input.h"
+#include "alias.h"
+#include "symtab.h"
+#include "wide-int.h"
+#include "inchash.h"
+#include "tree-core.h"
+#include "tree.h"
+#include "diagnostic-core.h"
+#include "gcc-rich-location.h"
+#include "print-tree.h"
+#include "pretty-print.h"
+#include "intl.h"
+#include "cpplib.h"
+#include "diagnostic.h"
+
+/* Extract any source range information from EXPR and write it
+   to *R.  */
+
+static bool
+get_range_for_expr (tree expr, location_range *r)
+{
+  if (EXPR_HAS_RANGE (expr))
+    {
+      source_range sr = EXPR_LOCATION_RANGE (expr);
+
+      /* Do we have meaningful data?  */
+      if (sr.m_start && sr.m_finish)
+	{
+	  r->m_start = expand_location (sr.m_start);
+	  r->m_finish = expand_location (sr.m_finish);
+	  return true;
+	}
+    }
+
+  return false;
+}
+
+/* Add a range to the rich_location, covering expression EXPR. */
+
+void
+gcc_rich_location::add_expr (tree expr)
+{
+  gcc_assert (expr);
+
+  location_range r;
+  r.m_show_caret_p = false;
+  if (get_range_for_expr (expr, &r))
+    add_range (&r);
+}
+
+/* If T is an expression, add a range for it to the rich_location.  */
+
+void
+gcc_rich_location::maybe_add_expr (tree t)
+{
+  if (EXPR_P (t))
+    add_expr (t);
+}
diff --git a/gcc/gcc-rich-location.h b/gcc/gcc-rich-location.h
new file mode 100644
index 0000000..2f9291d
--- /dev/null
+++ b/gcc/gcc-rich-location.h
@@ -0,0 +1,47 @@
+/* Declarations relating to class gcc_rich_location
+   Copyright (C) 2014-2015 Free Software Foundation, Inc.
+
+This file is part of GCC.
+
+GCC is free software; you can redistribute it and/or modify it under
+the terms of the GNU General Public License as published by the Free
+Software Foundation; either version 3, or (at your option) any later
+version.
+
+GCC is distributed in the hope that it will be useful, but WITHOUT ANY
+WARRANTY; without even the implied warranty of MERCHANTABILITY or
+FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License
+for more details.
+
+You should have received a copy of the GNU General Public License
+along with GCC; see the file COPYING3.  If not see
+<http://www.gnu.org/licenses/>.  */
+
+#ifndef GCC_RICH_LOCATION_H
+#define GCC_RICH_LOCATION_H
+
+/* A gcc_rich_location is libcpp's rich_location with additional
+   helper methods for working with gcc's types.  */
+class gcc_rich_location : public rich_location
+{
+ public:
+  /* Constructors.  */
+
+  /* Constructing from a location.  */
+  gcc_rich_location (source_location loc) :
+    rich_location (line_table, loc) {}
+
+  /* Constructing from a source_range.  */
+  gcc_rich_location (source_range src_range) :
+    rich_location (src_range) {}
+
+
+  /* Methods for adding ranges via gcc entities.  */
+  void
+  add_expr (tree expr);
+
+  void
+  maybe_add_expr (tree t);
+};
+
+#endif /* GCC_RICH_LOCATION_H */
diff --git a/gcc/genmatch.c b/gcc/genmatch.c
index 1eb8c24..9d74ed7 100644
--- a/gcc/genmatch.c
+++ b/gcc/genmatch.c
@@ -119,7 +119,7 @@ __attribute__((format (printf, 2, 3)))
 #endif
 fatal_at (const cpp_token *tk, const char *msg, ...)
 {
-  rich_location richloc (tk->src_loc);
+  rich_location richloc (line_table, tk->src_loc);
   va_list ap;
   va_start (ap, msg);
   error_cb (NULL, CPP_DL_FATAL, 0, &richloc, msg, &ap);
@@ -132,7 +132,7 @@ __attribute__((format (printf, 2, 3)))
 #endif
 fatal_at (source_location loc, const char *msg, ...)
 {
-  rich_location richloc (loc);
+  rich_location richloc (line_table, loc);
   va_list ap;
   va_start (ap, msg);
   error_cb (NULL, CPP_DL_FATAL, 0, &richloc, msg, &ap);
@@ -145,7 +145,7 @@ __attribute__((format (printf, 2, 3)))
 #endif
 warning_at (const cpp_token *tk, const char *msg, ...)
 {
-  rich_location richloc (tk->src_loc);
+  rich_location richloc (line_table, tk->src_loc);
   va_list ap;
   va_start (ap, msg);
   error_cb (NULL, CPP_DL_WARNING, 0, &richloc, msg, &ap);
@@ -158,7 +158,7 @@ __attribute__((format (printf, 2, 3)))
 #endif
 warning_at (source_location loc, const char *msg, ...)
 {
-  rich_location richloc (loc);
+  rich_location richloc (line_table, loc);
   va_list ap;
   va_start (ap, msg);
   error_cb (NULL, CPP_DL_WARNING, 0, &richloc, msg, &ap);
diff --git a/gcc/gimple.h b/gcc/gimple.h
index 781801b..8c60c47 100644
--- a/gcc/gimple.h
+++ b/gcc/gimple.h
@@ -1739,11 +1739,7 @@ gimple_block (const gimple *g)
 static inline void
 gimple_set_block (gimple *g, tree block)
 {
-  if (block)
-    g->location =
-	COMBINE_LOCATION_DATA (line_table, g->location, block);
-  else
-    g->location = LOCATION_LOCUS (g->location);
+  g->location = set_block (g->location, block);
 }
 
 
diff --git a/gcc/input.c b/gcc/input.c
index 0f6d448..ce84f10 100644
--- a/gcc/input.c
+++ b/gcc/input.c
@@ -887,6 +887,10 @@ dump_line_table_statistics (void)
 	   STAT_LABEL (s.adhoc_table_size));
   fprintf (stderr, "Ad-hoc table entries used:           %5ld\n",
 	   s.adhoc_table_entries_used);
+  fprintf (stderr, "optimized_ranges: %i\n",
+	   line_table->num_optimized_ranges);
+  fprintf (stderr, "unoptimized_ranges: %i\n",
+	   line_table->num_unoptimized_ranges);
 
   fprintf (stderr, "\n");
 }
@@ -917,13 +921,14 @@ write_digit (FILE *stream, int digit)
 
 static void
 write_digit_row (FILE *stream, int indent,
+		 const line_map_ordinary *map,
 		 source_location loc, int max_col, int divisor)
 {
   fprintf (stream, "%*c", indent, ' ');
   fprintf (stream, "|");
   for (int column = 1; column < max_col; column++)
     {
-      source_location column_loc = loc + column;
+      source_location column_loc = loc + (column << map->m_range_bits);
       write_digit (stream, column_loc / divisor);
     }
   fprintf (stream, "\n");
@@ -977,14 +982,20 @@ dump_location_info (FILE *stream)
       fprintf (stream, "  file: %s\n", ORDINARY_MAP_FILE_NAME (map));
       fprintf (stream, "  starting at line: %i\n",
 	       ORDINARY_MAP_STARTING_LINE_NUMBER (map));
+      fprintf (stream, "  column and range bits: %i\n",
+	       map->m_column_and_range_bits);
       fprintf (stream, "  column bits: %i\n",
-	       ORDINARY_MAP_NUMBER_OF_COLUMN_BITS (map));
+	       map->m_column_and_range_bits - map->m_range_bits);
+      fprintf (stream, "  range bits: %i\n",
+	       map->m_range_bits);
 
       /* Render the span of source lines that this "map" covers.  */
       for (source_location loc = MAP_START_LOCATION (map);
 	   loc < end_location;
-	   loc++)
+	   loc += (1 << map->m_range_bits) )
 	{
+	  gcc_assert (pure_location_p (line_table, loc) );
+
 	  expanded_location exploc
 	    = linemap_expand_location (line_table, map, loc);
 
@@ -1008,8 +1019,7 @@ dump_location_info (FILE *stream)
 		 Render the locations *within* the line, by underlining
 		 it, showing the source_location numeric values
 		 at each column.  */
-	      int max_col
-		= (1 << ORDINARY_MAP_NUMBER_OF_COLUMN_BITS (map)) - 1;
+	      int max_col = (1 << map->m_column_and_range_bits) - 1;
 	      if (max_col > line_size)
 		max_col = line_size + 1;
 
@@ -1017,17 +1027,17 @@ dump_location_info (FILE *stream)
 
 	      /* Thousands.  */
 	      if (end_location > 999)
-		write_digit_row (stream, indent, loc, max_col, 1000);
+		write_digit_row (stream, indent, map, loc, max_col, 1000);
 
 	      /* Hundreds.  */
 	      if (end_location > 99)
-		write_digit_row (stream, indent, loc, max_col, 100);
+		write_digit_row (stream, indent, map, loc, max_col, 100);
 
 	      /* Tens.  */
-	      write_digit_row (stream, indent, loc, max_col, 10);
+	      write_digit_row (stream, indent, map, loc, max_col, 10);
 
 	      /* Units.  */
-	      write_digit_row (stream, indent, loc, max_col, 1);
+	      write_digit_row (stream, indent, map, loc, max_col, 1);
 	    }
 	}
       fprintf (stream, "\n");
diff --git a/gcc/print-tree.c b/gcc/print-tree.c
index 1b584b8..cb0f1fd 100644
--- a/gcc/print-tree.c
+++ b/gcc/print-tree.c
@@ -938,6 +938,27 @@ print_node (FILE *file, const char *prefix, tree node, int indent)
       expanded_location xloc = expand_location (EXPR_LOCATION (node));
       indent_to (file, indent+4);
       fprintf (file, "%s:%d:%d", xloc.file, xloc.line, xloc.column);
+
+      /* Print the range, if any */
+      source_range r = EXPR_LOCATION_RANGE (node);
+      if (r.m_start)
+	{
+	  xloc = expand_location (r.m_start);
+	  fprintf (file, " start: %s:%d:%d", xloc.file, xloc.line, xloc.column);
+	}
+      else
+	{
+	  fprintf (file, " start: unknown");
+	}
+      if (r.m_finish)
+	{
+	  xloc = expand_location (r.m_finish);
+	  fprintf (file, " finish: %s:%d:%d", xloc.file, xloc.line, xloc.column);
+	}
+      else
+	{
+	  fprintf (file, " finish: unknown");
+	}
     }
 
   fprintf (file, ">");
diff --git a/gcc/rtl-error.c b/gcc/rtl-error.c
index 96da2bd..088bb8a 100644
--- a/gcc/rtl-error.c
+++ b/gcc/rtl-error.c
@@ -67,7 +67,7 @@ diagnostic_for_asm (const rtx_insn *insn, const char *msg, va_list *args_ptr,
 		    diagnostic_t kind)
 {
   diagnostic_info diagnostic;
-  rich_location richloc (location_for_asm (insn));
+  rich_location richloc (line_table, location_for_asm (insn));
 
   diagnostic_set_info (&diagnostic, msg, args_ptr,
 		       &richloc, kind);
diff --git a/gcc/testsuite/gcc.dg/diagnostic-token-ranges.c b/gcc/testsuite/gcc.dg/diagnostic-token-ranges.c
new file mode 100644
index 0000000..ac969e3
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/diagnostic-token-ranges.c
@@ -0,0 +1,120 @@
+/* { dg-options "-fdiagnostics-show-caret -Wc++-compat" } */
+
+/* Verify that various diagnostics show source code ranges.  */
+
+/* These ones merely use token ranges; they don't use tree ranges.  */
+
+void undeclared_identifier (void)
+{
+  name; /* { dg-error "'name' undeclared" } */
+/*
+{ dg-begin-multiline-output "" }
+   name;
+   ^~~~
+{ dg-end-multiline-output "" }
+*/
+}
+
+void unknown_type_name (void)
+{
+  foo bar; /* { dg-error "unknown type name 'foo'" } */
+/*
+{ dg-begin-multiline-output "" }
+   foo bar;
+   ^~~
+{ dg-end-multiline-output "" }
+*/
+
+  qux *baz; /* { dg-error "unknown type name 'qux'" } */
+/*
+{ dg-begin-multiline-output "" }
+   qux *baz;
+   ^~~
+{ dg-end-multiline-output "" }
+*/
+}
+
+void test_identifier_conflicts_with_cplusplus (void)
+{
+  int new; /* { dg-warning "identifier 'new' conflicts with" } */
+/*
+{ dg-begin-multiline-output "" }
+   int new;
+       ^~~
+{ dg-end-multiline-output "" }
+*/
+}
+
+extern void
+bogus_varargs (...); /* { dg-error "ISO C requires a named argument before '...'" } */
+/*
+{ dg-begin-multiline-output "" }
+ bogus_varargs (...);
+                ^~~
+{ dg-end-multiline-output "" }
+*/
+
+extern void
+foo (unknown_type param); /* { dg-error "unknown type name 'unknown_type'" } */
+/*
+{ dg-begin-multiline-output "" }
+ foo (unknown_type param);
+      ^~~~~~~~~~~~
+{ dg-end-multiline-output "" }
+*/
+
+void wide_string_literal_in_asm (void)
+{
+  asm (L"nop"); /* { dg-error "wide string literal in 'asm'" } */
+/*
+{ dg-begin-multiline-output "" }
+   asm (L"nop");
+        ^~~~~~
+{ dg-end-multiline-output "" }
+*/
+}
+
+void break_and_continue_in_wrong_places (void)
+{
+  if (0)
+    break; /* { dg-error "break statement not within loop or switch" } */
+/* { dg-begin-multiline-output "" }
+     break;
+     ^~~~~
+   { dg-end-multiline-output "" } */
+
+  if (1)
+    ;
+  else
+    continue; /* { dg-error "continue statement not within a loop" } */
+/* { dg-begin-multiline-output "" }
+     continue;
+     ^~~~~~~~
+    { dg-end-multiline-output "" } */
+}
+
+/* Various examples of bad type decls.  */
+
+int float bogus; /* { dg-error "two or more data types in declaration specifiers" } */
+/* { dg-begin-multiline-output "" }
+ int float bogus;
+     ^~~~~
+    { dg-end-multiline-output "" } */
+
+long long long bogus2; /* { dg-error "'long long long' is too long for GCC" } */
+/* { dg-begin-multiline-output "" }
+ long long long bogus2;
+           ^~~~
+    { dg-end-multiline-output "" } */
+
+long short bogus3; /* { dg-error "both 'long' and 'short' in declaration specifiers" } */
+/* { dg-begin-multiline-output "" }
+ long short bogus3;
+      ^~~~~
+    { dg-end-multiline-output "" } */
+
+signed unsigned bogus4; /* { dg-error "both 'signed' and 'unsigned' in declaration specifiers" } */
+/* { dg-begin-multiline-output "" }
+ signed unsigned bogus4;
+        ^~~~~~~~
+    { dg-end-multiline-output "" } */
diff --git a/gcc/testsuite/gcc.dg/diagnostic-tree-expr-ranges-2.c b/gcc/testsuite/gcc.dg/diagnostic-tree-expr-ranges-2.c
new file mode 100644
index 0000000..302e233
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/diagnostic-tree-expr-ranges-2.c
@@ -0,0 +1,23 @@
+/* { dg-do compile } */
+/* { dg-options "-Wuninitialized -fdiagnostics-show-caret" } */
+
+int test_uninit_1 (void)
+{
+  int result;
+  return result;  /* { dg-warning "uninitialized" } */
+/* { dg-begin-multiline-output "" }
+   return result;
+          ^~~~~~
+   { dg-end-multiline-output "" } */
+}
+
+int test_uninit_2 (void)
+{
+  int result;
+  result += 3; /* { dg-warning "uninitialized" } */
+/* { dg-begin-multiline-output "" }
+   result += 3;
+   ~~~~~~~^~~~
+   { dg-end-multiline-output "" } */
+  return result;
+}
diff --git a/gcc/testsuite/gcc.dg/plugin/diagnostic-test-expressions-1.c b/gcc/testsuite/gcc.dg/plugin/diagnostic-test-expressions-1.c
new file mode 100644
index 0000000..5485aaf
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/plugin/diagnostic-test-expressions-1.c
@@ -0,0 +1,422 @@
+/* { dg-do compile } */
+/* { dg-options "-O -fdiagnostics-show-caret" } */
+
+/* This is a collection of unittests to verify that we're correctly
+   capturing the source code ranges of various kinds of expression.
+
+   It uses the various "diagnostic_test_*_expression_range_plugin"
+   plugins which handles "__emit_expression_range" by generating a warning
+   at the given source range of the input argument.  Each of the
+   different plugins do this at a different phase of the internal
+   representation (tree, gimple, etc), so we can verify that the
+   source code range information is valid at each phase.
+
+   We want to accept an expression of any type.  To do this in C, we
+   use variadic arguments, but C requires at least one argument before
+   the ellipsis, so we have a dummy one.  */
+
+extern void __emit_expression_range (int dummy, ...);
+
+int global;
+
+void test_parentheses (int a, int b)
+{
+  __emit_expression_range (0, (a + b) ); /* { dg-warning "range" } */
+/* { dg-begin-multiline-output "" }
+   __emit_expression_range (0, (a + b) );
+                               ~~~^~~~
+   { dg-end-multiline-output "" } */
+
+  __emit_expression_range (0, (a + b) * (a - b) ); /* { dg-warning "range" } */
+/* { dg-begin-multiline-output "" }
+   __emit_expression_range (0, (a + b) * (a - b) );
+                               ~~~~~~~~^~~~~~~~~
+   { dg-end-multiline-output "" } */
+
+  __emit_expression_range (0, !(a && b) ); /* { dg-warning "range" } */
+/* { dg-begin-multiline-output "" }
+   __emit_expression_range (0, !(a && b) );
+                               ^~~~~~~~~
+   { dg-end-multiline-output "" } */
+}
+
+/* Postfix expressions.  ************************************************/
+
+void test_array_reference (int *arr)
+{
+  __emit_expression_range (0, arr[100] ); /* { dg-warning "range" } */
+/* { dg-begin-multiline-output "" }
+   __emit_expression_range (0, arr[100] );
+                               ~~~^~~~~
+   { dg-end-multiline-output "" } */
+}
+
+int test_function_call (int p, int q, int r)
+{
+  __emit_expression_range (0, test_function_call (p, q, r) ); /* { dg-warning "range" } */
+/* { dg-begin-multiline-output "" }
+   __emit_expression_range (0, test_function_call (p, q, r) );
+                               ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
+   { dg-end-multiline-output "" } */
+  return 0;
+}
+
+struct test_struct
+{
+  int field;
+};
+
+int test_structure_references (struct test_struct *ptr)
+{
+  struct test_struct local;
+  local.field = 42;
+
+  __emit_expression_range (0, local.field ); /* { dg-warning "range" } */
+/* { dg-begin-multiline-output "" }
+   __emit_expression_range (0, local.field );
+                               ~~~~~^~~~~~
+   { dg-end-multiline-output "" } */
+
+  __emit_expression_range (0, ptr->field ); /* { dg-warning "range" } */
+/* { dg-begin-multiline-output "" }
+   __emit_expression_range (0, ptr->field );
+                               ~~~^~~~~~~
+   { dg-end-multiline-output "" } */
+}
+
+int test_postfix_incdec (int i)
+{
+  __emit_expression_range (0, i++ ); /* { dg-warning "range" } */
+/* { dg-begin-multiline-output "" }
+   __emit_expression_range (0, i++ );
+                               ~^~
+   { dg-end-multiline-output "" } */
+
+  __emit_expression_range (0, i-- ); /* { dg-warning "range" } */
+/* { dg-begin-multiline-output "" }
+   __emit_expression_range (0, i-- );
+                               ~^~
+   { dg-end-multiline-output "" } */
+}
+
+/* Unary operators.  ****************************************************/
+
+int test_prefix_incdec (int i)
+{
+  __emit_expression_range (0, ++i ); /* { dg-warning "range" } */
+/* { dg-begin-multiline-output "" }
+   __emit_expression_range (0, ++i );
+                               ^~~
+   { dg-end-multiline-output "" } */
+
+  __emit_expression_range (0, --i ); /* { dg-warning "range" } */
+/* { dg-begin-multiline-output "" }
+   __emit_expression_range (0, --i );
+                               ^~~
+   { dg-end-multiline-output "" } */
+}
+
+void test_address_operator (void)
+{
+  __emit_expression_range (0, &global ); /* { dg-warning "range" } */
+/* { dg-begin-multiline-output "" }
+   __emit_expression_range (0, &global );
+                               ^~~~~~~
+   { dg-end-multiline-output "" } */
+}
+
+void test_indirection (int *ptr)
+{
+  __emit_expression_range (0, *ptr ); /* { dg-warning "range" } */
+/* { dg-begin-multiline-output "" }
+   __emit_expression_range (0, *ptr );
+                               ^~~~
+   { dg-end-multiline-output "" } */
+}
+
+void test_unary_minus (int i)
+{
+  __emit_expression_range (0, -i ); /* { dg-warning "range" } */
+/* { dg-begin-multiline-output "" }
+   __emit_expression_range (0, -i );
+                               ^~
+   { dg-end-multiline-output "" } */
+}
+
+void test_ones_complement (int i)
+{
+  __emit_expression_range (0, ~i ); /* { dg-warning "range" } */
+/* { dg-begin-multiline-output "" }
+   __emit_expression_range (0, ~i );
+                               ^~
+   { dg-end-multiline-output "" } */
+}
+
+void test_logical_negation (int flag)
+{
+  __emit_expression_range (0, !flag ); /* { dg-warning "range" } */
+/* { dg-begin-multiline-output "" }
+   __emit_expression_range (0, !flag );
+                               ^~~~~
+   { dg-end-multiline-output "" } */
+}
+
+/* Casts.  ****************************************************/
+
+void test_cast (void *ptr)
+{
+  __emit_expression_range (0, (int *)ptr ); /* { dg-warning "range" } */
+/* { dg-begin-multiline-output "" }
+   __emit_expression_range (0, (int *)ptr );
+                               ^~~~~~~~~~
+   { dg-end-multiline-output "" } */
+
+}
+
+/* Binary operators.  *******************************************/
+
+void test_multiplicative_operators (int lhs, int rhs)
+{
+  __emit_expression_range (0, lhs * rhs ); /* { dg-warning "range" } */
+/* { dg-begin-multiline-output "" }
+   __emit_expression_range (0, lhs * rhs );
+                               ~~~~^~~~~
+   { dg-end-multiline-output "" } */
+
+  __emit_expression_range (0, lhs / rhs ); /* { dg-warning "range" } */
+/* { dg-begin-multiline-output "" }
+   __emit_expression_range (0, lhs / rhs );
+                               ~~~~^~~~~
+   { dg-end-multiline-output "" } */
+
+  __emit_expression_range (0, lhs % rhs ); /* { dg-warning "range" } */
+/* { dg-begin-multiline-output "" }
+   __emit_expression_range (0, lhs % rhs );
+                               ~~~~^~~~~
+   { dg-end-multiline-output "" } */
+}
+
+void test_additive_operators (int lhs, int rhs)
+{
+  __emit_expression_range (0, lhs + rhs ); /* { dg-warning "range" } */
+/* { dg-begin-multiline-output "" }
+   __emit_expression_range (0, lhs + rhs );
+                               ~~~~^~~~~
+   { dg-end-multiline-output "" } */
+
+  __emit_expression_range (0, lhs - rhs ); /* { dg-warning "range" } */
+/* { dg-begin-multiline-output "" }
+   __emit_expression_range (0, lhs - rhs );
+                               ~~~~^~~~~
+   { dg-end-multiline-output "" } */
+}
+
+void test_shift_operators (int lhs, int rhs)
+{
+  __emit_expression_range (0, lhs << rhs ); /* { dg-warning "range" } */
+/* { dg-begin-multiline-output "" }
+   __emit_expression_range (0, lhs << rhs );
+                               ~~~~^~~~~~
+   { dg-end-multiline-output "" } */
+
+  __emit_expression_range (0, lhs >> rhs ); /* { dg-warning "range" } */
+/* { dg-begin-multiline-output "" }
+   __emit_expression_range (0, lhs >> rhs );
+                               ~~~~^~~~~~
+   { dg-end-multiline-output "" } */
+}
+
+void test_relational_operators (int lhs, int rhs)
+{
+  __emit_expression_range (0, lhs < rhs ); /* { dg-warning "range" } */
+/* { dg-begin-multiline-output "" }
+   __emit_expression_range (0, lhs < rhs );
+                               ~~~~^~~~~
+   { dg-end-multiline-output "" } */
+
+  __emit_expression_range (0, lhs > rhs ); /* { dg-warning "range" } */
+/* { dg-begin-multiline-output "" }
+   __emit_expression_range (0, lhs > rhs );
+                               ~~~~^~~~~
+   { dg-end-multiline-output "" } */
+
+  __emit_expression_range (0, lhs <= rhs ); /* { dg-warning "range" } */
+/* { dg-begin-multiline-output "" }
+   __emit_expression_range (0, lhs <= rhs );
+                               ~~~~^~~~~~
+   { dg-end-multiline-output "" } */
+
+  __emit_expression_range (0, lhs >= rhs ); /* { dg-warning "range" } */
+/* { dg-begin-multiline-output "" }
+   __emit_expression_range (0, lhs >= rhs );
+                               ~~~~^~~~~~
+   { dg-end-multiline-output "" } */
+}
+
+void test_equality_operators (int lhs, int rhs)
+{
+  __emit_expression_range (0, lhs == rhs ); /* { dg-warning "range" } */
+/* { dg-begin-multiline-output "" }
+   __emit_expression_range (0, lhs == rhs );
+                               ~~~~^~~~~~
+   { dg-end-multiline-output "" } */
+
+  __emit_expression_range (0, lhs != rhs ); /* { dg-warning "range" } */
+/* { dg-begin-multiline-output "" }
+   __emit_expression_range (0, lhs != rhs );
+                               ~~~~^~~~~~
+   { dg-end-multiline-output "" } */
+}
+
+void test_bitwise_binary_operators (int lhs, int rhs)
+{
+  __emit_expression_range (0, lhs & rhs ); /* { dg-warning "range" } */
+/* { dg-begin-multiline-output "" }
+   __emit_expression_range (0, lhs & rhs );
+                               ~~~~^~~~~
+   { dg-end-multiline-output "" } */
+
+  __emit_expression_range (0, lhs ^ rhs ); /* { dg-warning "range" } */
+/* { dg-begin-multiline-output "" }
+   __emit_expression_range (0, lhs ^ rhs );
+                               ~~~~^~~~~
+   { dg-end-multiline-output "" } */
+
+  __emit_expression_range (0, lhs | rhs ); /* { dg-warning "range" } */
+/* { dg-begin-multiline-output "" }
+   __emit_expression_range (0, lhs | rhs );
+                               ~~~~^~~~~
+   { dg-end-multiline-output "" } */
+}
+
+void test_logical_operators (int lhs, int rhs)
+{
+  __emit_expression_range (0, lhs && rhs ); /* { dg-warning "range" } */
+/* { dg-begin-multiline-output "" }
+   __emit_expression_range (0, lhs && rhs );
+                               ~~~~^~~~~~
+   { dg-end-multiline-output "" } */
+
+  __emit_expression_range (0, lhs || rhs ); /* { dg-warning "range" } */
+/* { dg-begin-multiline-output "" }
+   __emit_expression_range (0, lhs || rhs );
+                               ~~~~^~~~~~
+   { dg-end-multiline-output "" } */
+}
+
+/* Conditional operator.  *******************************************/
+
+void test_conditional_operators (int flag, int on_true, int on_false)
+{
+  __emit_expression_range (0, flag ? on_true : on_false ); /* { dg-warning "range" } */
+/* { dg-begin-multiline-output "" }
+   __emit_expression_range (0, flag ? on_true : on_false );
+                               ~~~~~~~~~~~~~~~^~~~~~~~~~
+   { dg-end-multiline-output "" } */
+}
+
+/* Assignment expressions.  *******************************************/
+
+void test_assignment_expressions (int dest, int other)
+{
+  __emit_expression_range (0, dest = other ); /* { dg-warning "range" } */
+/* { dg-begin-multiline-output "" }
+   __emit_expression_range (0, dest = other );
+                               ~~~~~^~~~~~~
+   { dg-end-multiline-output "" } */
+
+  __emit_expression_range (0, dest *= other ); /* { dg-warning "range" } */
+/* { dg-begin-multiline-output "" }
+   __emit_expression_range (0, dest *= other );
+                               ~~~~~^~~~~~~~
+   { dg-end-multiline-output "" } */
+
+  __emit_expression_range (0, dest /= other ); /* { dg-warning "range" } */
+/* { dg-begin-multiline-output "" }
+   __emit_expression_range (0, dest /= other );
+                               ~~~~~^~~~~~~~
+   { dg-end-multiline-output "" } */
+
+  __emit_expression_range (0, dest %= other ); /* { dg-warning "range" } */
+/* { dg-begin-multiline-output "" }
+   __emit_expression_range (0, dest %= other );
+                               ~~~~~^~~~~~~~
+   { dg-end-multiline-output "" } */
+
+  __emit_expression_range (0, dest += other ); /* { dg-warning "range" } */
+/* { dg-begin-multiline-output "" }
+   __emit_expression_range (0, dest += other );
+                               ~~~~~^~~~~~~~
+   { dg-end-multiline-output "" } */
+
+  __emit_expression_range (0, dest -= other ); /* { dg-warning "range" } */
+/* { dg-begin-multiline-output "" }
+   __emit_expression_range (0, dest -= other );
+                               ~~~~~^~~~~~~~
+   { dg-end-multiline-output "" } */
+
+  __emit_expression_range (0, dest <<= other ); /* { dg-warning "range" } */
+/* { dg-begin-multiline-output "" }
+   __emit_expression_range (0, dest <<= other );
+                               ~~~~~^~~~~~~~~
+   { dg-end-multiline-output "" } */
+
+  __emit_expression_range (0, dest >>= other ); /* { dg-warning "range" } */
+/* { dg-begin-multiline-output "" }
+   __emit_expression_range (0, dest >>= other );
+                               ~~~~~^~~~~~~~~
+   { dg-end-multiline-output "" } */
+
+  __emit_expression_range (0, dest &= other ); /* { dg-warning "range" } */
+/* { dg-begin-multiline-output "" }
+   __emit_expression_range (0, dest &= other );
+                               ~~~~~^~~~~~~~
+   { dg-end-multiline-output "" } */
+
+  __emit_expression_range (0, dest ^= other ); /* { dg-warning "range" } */
+/* { dg-begin-multiline-output "" }
+   __emit_expression_range (0, dest ^= other );
+                               ~~~~~^~~~~~~~
+   { dg-end-multiline-output "" } */
+
+  __emit_expression_range (0, dest |= other ); /* { dg-warning "range" } */
+/* { dg-begin-multiline-output "" }
+   __emit_expression_range (0, dest |= other );
+                               ~~~~~^~~~~~~~
+   { dg-end-multiline-output "" } */
+}
+
+/* Comma operator.  *******************************************/
+
+void test_comma_operator (int a, int b)
+{
+  __emit_expression_range (0, (a++, a + b) ); /* { dg-warning "range" } */
+/* { dg-begin-multiline-output "" }
+   __emit_expression_range (0, (a++, a + b) );
+                               ~~~~^~~~~~~~
+   { dg-end-multiline-output "" } */
+}
+
+/* Examples of non-trivial expressions.  ****************************/
+
+extern double sqrt (double x);
+
+void test_quadratic (double a, double b, double c)
+{
+  __emit_expression_range (0, b * b - 4 * a * c ); /* { dg-warning "range" } */
+/* { dg-begin-multiline-output "" }
+   __emit_expression_range (0, b * b - 4 * a * c );
+                               ~~~~~~^~~~~~~~~~~
+   { dg-end-multiline-output "" } */
+
+  __emit_expression_range (0,
+     (-b + sqrt (b * b - 4 * a * c))
+     / (2 * a)); /* { dg-warning "range" } */
+/* { dg-begin-multiline-output "" }
+      (-b + sqrt (b * b - 4 * a * c))
+      ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+      / (2 * a));
+      ^~~~~~~~~
+   { dg-end-multiline-output "" } */
+
+}
diff --git a/gcc/testsuite/gcc.dg/plugin/diagnostic-test-show-trees-1.c b/gcc/testsuite/gcc.dg/plugin/diagnostic-test-show-trees-1.c
new file mode 100644
index 0000000..7473a07
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/plugin/diagnostic-test-show-trees-1.c
@@ -0,0 +1,65 @@
+/* { dg-do compile } */
+/* { dg-options "-O -fdiagnostics-show-caret" } */
+
+/* This is an example file for use with
+   diagnostic_plugin_show_trees.c.
+
+   The plugin handles "__show_tree" by recursively dumping
+   the internal structure of the second input argument.
+
+   We want to accept an expression of any type.  To do this in C, we
+   use variadic arguments, but C requires at least one argument before
+   the ellipsis, so we have a dummy one.  */
+
+extern void __show_tree (int dummy, ...);
+
+extern double sqrt (double x);
+
+void test_quadratic (double a, double b, double c)
+{
+  __show_tree (0,
+     (-b + sqrt (b * b - 4 * a * c))
+     / (2 * a));
+
+/* { dg-begin-multiline-output "" }
+      (-b + sqrt (b * b - 4 * a * c))
+      ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+      / (2 * a));
+      ^~~~~~~~~
+   { dg-end-multiline-output "" } */
+
+/* { dg-begin-multiline-output "" }
+      (-b + sqrt (b * b - 4 * a * c))
+      ~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~
+   { dg-end-multiline-output "" } */
+
+/* { dg-begin-multiline-output "" }
+      (-b + sqrt (b * b - 4 * a * c))
+            ^~~~~~~~~~~~~~~~~~~~~~~~
+   { dg-end-multiline-output "" } */
+
+/* { dg-begin-multiline-output "" }
+      (-b + sqrt (b * b - 4 * a * c))
+                  ~~~~~~^~~~~~~~~~~
+   { dg-end-multiline-output "" } */
+
+/* { dg-begin-multiline-output "" }
+      (-b + sqrt (b * b - 4 * a * c))
+                  ~~^~~
+   { dg-end-multiline-output "" } */
+
+/* { dg-begin-multiline-output "" }
+      (-b + sqrt (b * b - 4 * a * c))
+                          ~~~~~~^~~
+   { dg-end-multiline-output "" } */
+
+/* { dg-begin-multiline-output "" }
+      (-b + sqrt (b * b - 4 * a * c))
+                          ~~^~~
+   { dg-end-multiline-output "" } */
+
+/* { dg-begin-multiline-output "" }
+      / (2 * a));
+        ~~~^~~~
+   { dg-end-multiline-output "" } */
+}
diff --git a/gcc/testsuite/gcc.dg/plugin/diagnostic_plugin_show_trees.c b/gcc/testsuite/gcc.dg/plugin/diagnostic_plugin_show_trees.c
new file mode 100644
index 0000000..5a911c1
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/plugin/diagnostic_plugin_show_trees.c
@@ -0,0 +1,174 @@
+/* This plugin recursively dumps the source-code location ranges of
+   expressions, at the pre-gimplification tree stage.  */
+/* { dg-options "-O" } */
+
+#include "gcc-plugin.h"
+#include "config.h"
+#include "system.h"
+#include "coretypes.h"
+#include "tm.h"
+#include "tree.h"
+#include "stringpool.h"
+#include "toplev.h"
+#include "basic-block.h"
+#include "hash-table.h"
+#include "vec.h"
+#include "ggc.h"
+#include "basic-block.h"
+#include "tree-ssa-alias.h"
+#include "internal-fn.h"
+#include "gimple-fold.h"
+#include "tree-eh.h"
+#include "gimple-expr.h"
+#include "is-a.h"
+#include "gimple.h"
+#include "gimple-iterator.h"
+#include "tree.h"
+#include "tree-pass.h"
+#include "intl.h"
+#include "plugin-version.h"
+#include "diagnostic.h"
+#include "context.h"
+#include "gcc-rich-location.h"
+#include "print-tree.h"
+
+/*
+  Hack: fails with linker error:
+./diagnostic_plugin_show_trees.so: undefined symbol: _ZN17gcc_rich_location8add_exprEP9tree_node
+  since nothing in the tree is using gcc_rich_location::add_expr yet.
+
+  I've tried various workarounds (adding DEBUG_FUNCTION to the
+  method, taking its address), but can't seem to fix it that way.
+  So as a nasty workaround, the following material is copied&pasted
+  from gcc-rich-location.c: */
+
+static bool
+get_range_for_expr (tree expr, location_range *r)
+{
+  if (EXPR_HAS_RANGE (expr))
+    {
+      source_range sr = EXPR_LOCATION_RANGE (expr);
+
+      /* Do we have meaningful data?  */
+      if (sr.m_start && sr.m_finish)
+	{
+	  r->m_start = expand_location (sr.m_start);
+	  r->m_finish = expand_location (sr.m_finish);
+	  return true;
+	}
+    }
+
+  return false;
+}
+
+/* Add a range to the rich_location, covering expression EXPR. */
+
+void
+gcc_rich_location::add_expr (tree expr)
+{
+  gcc_assert (expr);
+
+  location_range r;
+  r.m_show_caret_p = false;
+  if (get_range_for_expr (expr, &r))
+    add_range (&r);
+}
+
+/* FIXME: end of material taken from gcc-rich-location.c */
+
+int plugin_is_GPL_compatible;
+
+static void
+show_tree (tree node)
+{
+  if (!CAN_HAVE_RANGE_P (node))
+    return;
+
+  gcc_rich_location richloc (EXPR_LOCATION (node));
+  richloc.add_expr (node);
+
+  if (richloc.get_num_locations () < 2)
+    {
+      error_at_rich_loc (&richloc, "range not found");
+      return;
+    }
+
+  enum tree_code code = TREE_CODE (node);
+
+  location_range *range = richloc.get_range (1);
+  inform_at_rich_loc (&richloc,
+		      "%s at range %i:%i-%i:%i",
+		      get_tree_code_name (code),
+		      range->m_start.line,
+		      range->m_start.column,
+		      range->m_finish.line,
+		      range->m_finish.column);
+
+  /* Recurse.  */
+  int min_idx = 0;
+  int max_idx = TREE_OPERAND_LENGTH (node);
+  switch (code)
+    {
+    case CALL_EXPR:
+      min_idx = 3;
+      break;
+
+    default:
+      break;
+    }
+
+  for (int i = min_idx; i < max_idx; i++)
+    show_tree (TREE_OPERAND (node, i));
+}
+
+tree
+cb_walk_tree_fn (tree * tp, int * walk_subtrees,
+		 void * data ATTRIBUTE_UNUSED)
+{
+  if (TREE_CODE (*tp) != CALL_EXPR)
+    return NULL_TREE;
+
+  tree call_expr = *tp;
+  tree fn = CALL_EXPR_FN (call_expr);
+  if (TREE_CODE (fn) != ADDR_EXPR)
+    return NULL_TREE;
+  fn = TREE_OPERAND (fn, 0);
+  if (TREE_CODE (fn) != FUNCTION_DECL)
+    return NULL_TREE;
+  if (strcmp (IDENTIFIER_POINTER (DECL_NAME (fn)), "__show_tree"))
+    return NULL_TREE;
+
+  /* Get arg 1; print it! */
+  tree arg = CALL_EXPR_ARG (call_expr, 1);
+
+  show_tree (arg);
+
+  return NULL_TREE;
+}
+
+static void
+callback (void *gcc_data, void *user_data)
+{
+  tree fndecl = (tree)gcc_data;
+  walk_tree (&DECL_SAVED_TREE (fndecl), cb_walk_tree_fn, NULL, NULL);
+}
+
+int
+plugin_init (struct plugin_name_args *plugin_info,
+	     struct plugin_gcc_version *version)
+{
+  struct register_pass_info pass_info;
+  const char *plugin_name = plugin_info->base_name;
+  int argc = plugin_info->argc;
+  struct plugin_argument *argv = plugin_info->argv;
+
+  if (!plugin_default_version_check (version, &gcc_version))
+    return 1;
+
+  register_callback (plugin_name,
+		     PLUGIN_PRE_GENERICIZE,
+		     callback,
+		     NULL);
+
+  return 0;
+}
diff --git a/gcc/testsuite/gcc.dg/plugin/diagnostic_plugin_test_show_locus.c b/gcc/testsuite/gcc.dg/plugin/diagnostic_plugin_test_show_locus.c
index 8f5724e..158c612 100644
--- a/gcc/testsuite/gcc.dg/plugin/diagnostic_plugin_test_show_locus.c
+++ b/gcc/testsuite/gcc.dg/plugin/diagnostic_plugin_test_show_locus.c
@@ -109,7 +109,8 @@ get_loc (unsigned int line_num, unsigned int col_num)
 
   /* Convert from 0-based column numbers to 1-based column numbers.  */
   source_location loc
-    = linemap_position_for_line_and_column (line_map,
+    = linemap_position_for_line_and_column (line_table,
+					    line_map,
 					    line_num, col_num + 1);
 
   return loc;
@@ -163,7 +164,7 @@ test_show_locus (function *fun)
   if (0 == strcmp (fnname, "test_simple"))
     {
       const int line = fnstart_line + 2;
-      rich_location richloc (get_loc (line, 15));
+      rich_location richloc (line_table, get_loc (line, 15));
       richloc.add_range (get_loc (line, 10), get_loc (line, 14), false);
       richloc.add_range (get_loc (line, 16), get_loc (line, 16), false);
       warning_at_rich_loc (&richloc, 0, "test");
@@ -172,7 +173,7 @@ test_show_locus (function *fun)
   if (0 == strcmp (fnname, "test_simple_2"))
     {
       const int line = fnstart_line + 2;
-      rich_location richloc (get_loc (line, 24));
+      rich_location richloc (line_table, get_loc (line, 24));
       richloc.add_range (get_loc (line, 6),
 			 get_loc (line, 22), false);
       richloc.add_range (get_loc (line, 26),
@@ -183,7 +184,7 @@ test_show_locus (function *fun)
   if (0 == strcmp (fnname, "test_multiline"))
     {
       const int line = fnstart_line + 2;
-      rich_location richloc (get_loc (line + 1, 7));
+      rich_location richloc (line_table, get_loc (line + 1, 7));
       richloc.add_range (get_loc (line, 7),
 			 get_loc (line, 23), false);
       richloc.add_range (get_loc (line + 1, 9),
@@ -194,7 +195,7 @@ test_show_locus (function *fun)
   if (0 == strcmp (fnname, "test_many_lines"))
     {
       const int line = fnstart_line + 2;
-      rich_location richloc (get_loc (line + 5, 7));
+      rich_location richloc (line_table, get_loc (line + 5, 7));
       richloc.add_range (get_loc (line, 7),
 			 get_loc (line + 4, 65), false);
       richloc.add_range (get_loc (line + 5, 9),
@@ -223,7 +224,7 @@ test_show_locus (function *fun)
       source_range src_range;
       src_range.m_start = get_loc (line, 12);
       src_range.m_finish = get_loc (line, 20);
-      rich_location richloc (caret);
+      rich_location richloc (line_table, caret);
       richloc.set_range (0, src_range, true, false);
       warning_at_rich_loc (&richloc, 0, "test");
     }
@@ -237,7 +238,7 @@ test_show_locus (function *fun)
       source_range src_range;
       src_range.m_start = get_loc (line, 90);
       src_range.m_finish = get_loc (line, 98);
-      rich_location richloc (caret);
+      rich_location richloc (line_table, caret);
       richloc.set_range (0, src_range, true, false);
       warning_at_rich_loc (&richloc, 0, "test");
     }
@@ -248,7 +249,7 @@ test_show_locus (function *fun)
       const int line = fnstart_line + 2;
       location_t caret_a = get_loc (line, 7);
       location_t caret_b = get_loc (line, 11);
-      rich_location richloc (caret_a);
+      rich_location richloc (line_table, caret_a);
       richloc.add_range (caret_b, caret_b, true);
       global_dc->caret_chars[0] = 'A';
       global_dc->caret_chars[1] = 'B';
@@ -269,7 +270,7 @@ test_show_locus (function *fun)
       const int line = fnstart_line + 3;
       location_t caret_a = get_loc (line, 5);
       location_t caret_b = get_loc (line - 1, 19);
-      rich_location richloc (caret_a);
+      rich_location richloc (line_table, caret_a);
       richloc.add_range (caret_b, caret_b, true);
       global_dc->caret_chars[0] = '1';
       global_dc->caret_chars[1] = '2';
@@ -304,11 +305,6 @@ plugin_init (struct plugin_name_args *plugin_info,
   if (!plugin_default_version_check (version, &gcc_version))
     return 1;
 
-  /* For now, tell the dc to expect ranges and thus to colorize the source
-     lines, not just the carets/underlines.  This will be redundant
-     once the C frontend generates ranges.  */
-  global_dc->colorize_source_p = true;
-
   for (int i = 0; i < argc; i++)
     {
       if (0 == strcmp (argv[i].key, "color"))
diff --git a/gcc/testsuite/gcc.dg/plugin/diagnostic_plugin_test_tree_expression_range.c b/gcc/testsuite/gcc.dg/plugin/diagnostic_plugin_test_tree_expression_range.c
new file mode 100644
index 0000000..89cc95a
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/plugin/diagnostic_plugin_test_tree_expression_range.c
@@ -0,0 +1,98 @@
+/* This plugin verifies the source-code location ranges of
+   expressions, at the pre-gimplification tree stage.  */
+/* { dg-options "-O" } */
+
+#include "gcc-plugin.h"
+#include "config.h"
+#include "system.h"
+#include "coretypes.h"
+#include "tm.h"
+#include "tree.h"
+#include "stringpool.h"
+#include "toplev.h"
+#include "basic-block.h"
+#include "hash-table.h"
+#include "vec.h"
+#include "ggc.h"
+#include "basic-block.h"
+#include "tree-ssa-alias.h"
+#include "internal-fn.h"
+#include "gimple-fold.h"
+#include "tree-eh.h"
+#include "gimple-expr.h"
+#include "is-a.h"
+#include "gimple.h"
+#include "gimple-iterator.h"
+#include "tree.h"
+#include "tree-pass.h"
+#include "intl.h"
+#include "plugin-version.h"
+#include "diagnostic.h"
+#include "context.h"
+#include "print-tree.h"
+
+int plugin_is_GPL_compatible;
+
+static void
+emit_warning (location_t loc)
+{
+  source_range src_range = get_range_from_loc (line_table, loc);
+  warning_at (loc, 0,
+	      "tree range %i:%i-%i:%i",
+	      LOCATION_LINE (src_range.m_start),
+	      LOCATION_COLUMN (src_range.m_start),
+	      LOCATION_LINE (src_range.m_finish),
+	      LOCATION_COLUMN (src_range.m_finish));
+}
+
+tree
+cb_walk_tree_fn (tree * tp, int * walk_subtrees,
+		 void * data ATTRIBUTE_UNUSED)
+{
+  if (TREE_CODE (*tp) != CALL_EXPR)
+    return NULL_TREE;
+
+  tree call_expr = *tp;
+  tree fn = CALL_EXPR_FN (call_expr);
+  if (TREE_CODE (fn) != ADDR_EXPR)
+    return NULL_TREE;
+  fn = TREE_OPERAND (fn, 0);
+  if (TREE_CODE (fn) != FUNCTION_DECL)
+    return NULL_TREE;
+  if (strcmp (IDENTIFIER_POINTER (DECL_NAME (fn)), "__emit_expression_range"))
+    return NULL_TREE;
+
+  /* Get arg 1; print it! */
+  tree arg = CALL_EXPR_ARG (call_expr, 1);
+
+  emit_warning (EXPR_LOCATION (arg));
+
+  return NULL_TREE;
+}
+
+static void
+callback (void *gcc_data, void *user_data)
+{
+  tree fndecl = (tree)gcc_data;
+  walk_tree (&DECL_SAVED_TREE (fndecl), cb_walk_tree_fn, NULL, NULL);
+}
+
+int
+plugin_init (struct plugin_name_args *plugin_info,
+	     struct plugin_gcc_version *version)
+{
+  struct register_pass_info pass_info;
+  const char *plugin_name = plugin_info->base_name;
+  int argc = plugin_info->argc;
+  struct plugin_argument *argv = plugin_info->argv;
+
+  if (!plugin_default_version_check (version, &gcc_version))
+    return 1;
+
+  register_callback (plugin_name,
+		     PLUGIN_PRE_GENERICIZE,
+		     callback,
+		     NULL);
+
+  return 0;
+}
diff --git a/gcc/testsuite/gcc.dg/plugin/plugin.exp b/gcc/testsuite/gcc.dg/plugin/plugin.exp
index 941bccc..f1155ee 100644
--- a/gcc/testsuite/gcc.dg/plugin/plugin.exp
+++ b/gcc/testsuite/gcc.dg/plugin/plugin.exp
@@ -66,6 +66,10 @@ set plugin_test_list [list \
     { diagnostic_plugin_test_show_locus.c \
 	  diagnostic-test-show-locus-bw.c \
 	  diagnostic-test-show-locus-color.c } \
+    { diagnostic_plugin_test_tree_expression_range.c \
+	  diagnostic-test-expressions-1.c } \
+    { diagnostic_plugin_show_trees.c \
+	  diagnostic-test-show-trees-1.c } \
 ]
 
 foreach plugin_test $plugin_test_list {
diff --git a/gcc/toplev.c b/gcc/toplev.c
index 140e36f..588d89d 100644
--- a/gcc/toplev.c
+++ b/gcc/toplev.c
@@ -1130,6 +1130,7 @@ general_init (const char *argv0, bool init_signals)
   linemap_init (line_table, BUILTINS_LOCATION);
   line_table->reallocator = realloc_for_line_map;
   line_table->round_alloc_size = ggc_round_alloc_size;
+  line_table->default_range_bits = 5;
   init_ttree ();
 
   /* Initialize register usage now so switches may override.  */
diff --git a/gcc/tree-cfg.c b/gcc/tree-cfg.c
index 5d98eec..0c624aa 100644
--- a/gcc/tree-cfg.c
+++ b/gcc/tree-cfg.c
@@ -6719,10 +6719,7 @@ move_block_to_fn (struct function *dest_cfun, basic_block bb,
 	    continue;
 	  if (d->orig_block == NULL_TREE || block == d->orig_block)
 	    {
-	      if (d->new_block == NULL_TREE)
-		locus = LOCATION_LOCUS (locus);
-	      else
-		locus = COMBINE_LOCATION_DATA (line_table, locus, d->new_block);
+	      locus = set_block (locus, d->new_block);
 	      gimple_phi_arg_set_location (phi, i, locus);
 	    }
 	}
@@ -6782,9 +6779,7 @@ move_block_to_fn (struct function *dest_cfun, basic_block bb,
 	tree block = LOCATION_BLOCK (e->goto_locus);
 	if (d->orig_block == NULL_TREE
 	    || block == d->orig_block)
-	  e->goto_locus = d->new_block ?
-	      COMBINE_LOCATION_DATA (line_table, e->goto_locus, d->new_block) :
-	      LOCATION_LOCUS (e->goto_locus);
+	  e->goto_locus = set_block (e->goto_locus, d->new_block);
       }
 }
 
diff --git a/gcc/tree-inline.c b/gcc/tree-inline.c
index 17d97a8..205c869 100644
--- a/gcc/tree-inline.c
+++ b/gcc/tree-inline.c
@@ -2348,10 +2348,7 @@ copy_phis_for_bb (basic_block bb, copy_body_data *id)
 		  tree *n;
 		  n = id->decl_map->get (LOCATION_BLOCK (locus));
 		  gcc_assert (n);
-		  if (*n)
-		    locus = COMBINE_LOCATION_DATA (line_table, locus, *n);
-		  else
-		    locus = LOCATION_LOCUS (locus);
+		  locus = set_block (locus, *n);
 		}
 	      else
 		locus = LOCATION_LOCUS (locus);
diff --git a/gcc/tree.c b/gcc/tree.c
index 50e1db0..1d770c3 100644
--- a/gcc/tree.c
+++ b/gcc/tree.c
@@ -11789,10 +11789,7 @@ tree_set_block (tree t, tree b)
 
   if (IS_EXPR_CODE_CLASS (c))
     {
-      if (b)
-	t->exp.locus = COMBINE_LOCATION_DATA (line_table, t->exp.locus, b);
-      else
-	t->exp.locus = LOCATION_LOCUS (t->exp.locus);
+      t->exp.locus = set_block (t->exp.locus, b);
     }
   else
     gcc_unreachable ();
@@ -13813,5 +13810,60 @@ nonnull_arg_p (const_tree arg)
   return false;
 }
 
+/* Given location LOC, strip away any packed range information
+   or ad-hoc information.  */
+
+static location_t
+get_pure_location (location_t loc)
+{
+  if (IS_ADHOC_LOC (loc))
+    loc
+      = line_table->location_adhoc_data_map.data[loc & MAX_SOURCE_LOCATION].locus;
+
+  if (loc >= LINEMAPS_MACRO_LOWEST_LOCATION (line_table))
+    return loc;
+
+  if (loc < RESERVED_LOCATION_COUNT)
+    return loc;
+
+  const line_map *map = linemap_lookup (line_table, loc);
+  const line_map_ordinary *ordmap = linemap_check_ordinary (map);
+
+  return loc & ~((1 << ordmap->m_range_bits) - 1);
+}
+
+/* Combine LOC and BLOCK to a combined adhoc loc, retaining any range
+   information.  */
+
+location_t
+set_block (location_t loc, tree block)
+{
+  location_t pure_loc = get_pure_location (loc);
+  source_range src_range = get_range_from_loc (line_table, loc);
+  return COMBINE_LOCATION_DATA (line_table, pure_loc, src_range, block);
+}
+
+void
+set_source_range (tree expr, location_t start, location_t finish)
+{
+  source_range src_range;
+  src_range.m_start = start;
+  src_range.m_finish = finish;
+  set_source_range (expr, src_range);
+}
+
+void
+set_source_range (tree expr, source_range src_range)
+{
+  if (!EXPR_P (expr))
+    return;
+
+  location_t pure_loc = get_pure_location (EXPR_LOCATION (expr));
+  location_t adhoc = COMBINE_LOCATION_DATA (line_table,
+					    pure_loc,
+					    src_range,
+					    NULL);
+  SET_EXPR_LOCATION (expr, adhoc);
+}
 
 #include "gt-tree.h"
diff --git a/gcc/tree.h b/gcc/tree.h
index 1bb59f2..0b9c3b9 100644
--- a/gcc/tree.h
+++ b/gcc/tree.h
@@ -1096,10 +1096,25 @@ extern void omp_clause_range_check_failed (const_tree, const char *, int,
 #define EXPR_FILENAME(NODE) LOCATION_FILE (EXPR_CHECK ((NODE))->exp.locus)
 #define EXPR_LINENO(NODE) LOCATION_LINE (EXPR_CHECK (NODE)->exp.locus)
 
+#define CAN_HAVE_RANGE_P(NODE) (CAN_HAVE_LOCATION_P (NODE))
+#define EXPR_LOCATION_RANGE(NODE) (get_expr_source_range (EXPR_CHECK ((NODE))))
+
+#define EXPR_HAS_RANGE(NODE) \
+    (CAN_HAVE_RANGE_P (NODE) \
+     ? EXPR_LOCATION_RANGE (NODE).m_start != UNKNOWN_LOCATION \
+     : false)
+
 /* True if a tree is an expression or statement that can have a
    location.  */
 #define CAN_HAVE_LOCATION_P(NODE) ((NODE) && EXPR_P (NODE))
 
+static inline source_range
+get_expr_source_range (tree expr)
+{
+  location_t loc = EXPR_LOCATION (expr);
+  return get_range_from_loc (line_table, loc);
+}
+
 extern void protected_set_expr_location (tree, location_t);
 
 /* In a TARGET_EXPR node.  */
@@ -2172,6 +2187,9 @@ extern machine_mode element_mode (const_tree t);
 #define DECL_IS_BUILTIN(DECL) \
   (LOCATION_LOCUS (DECL_SOURCE_LOCATION (DECL)) <= BUILTINS_LOCATION)
 
+#define DECL_LOCATION_RANGE(NODE) \
+  (get_decl_source_range (DECL_MINIMAL_CHECK (NODE)))
+
 /*  For FIELD_DECLs, this is the RECORD_TYPE, UNION_TYPE, or
     QUAL_UNION_TYPE node that the field is a member of.  For VAR_DECL,
     PARM_DECL, FUNCTION_DECL, LABEL_DECL, RESULT_DECL, and CONST_DECL
@@ -5277,10 +5295,25 @@ type_with_alias_set_p (const_tree t)
   return false;
 }
 
+extern location_t set_block (location_t loc, tree block);
+
 extern void gt_ggc_mx (tree &);
 extern void gt_pch_nx (tree &);
 extern void gt_pch_nx (tree &, gt_pointer_operator, void *);
 
 extern bool nonnull_arg_p (const_tree);
 
+extern void
+set_source_range (tree expr, location_t start, location_t finish);
+
+extern void
+set_source_range (tree expr, source_range src_range);
+
+static inline source_range
+get_decl_source_range (tree decl)
+{
+  location_t loc = DECL_SOURCE_LOCATION (decl);
+  return get_range_from_loc (line_table, loc);
+}
+
 #endif  /* GCC_TREE_H  */
diff --git a/libcpp/errors.c b/libcpp/errors.c
index c351c11..8790e10 100644
--- a/libcpp/errors.c
+++ b/libcpp/errors.c
@@ -57,7 +57,7 @@ cpp_diagnostic (cpp_reader * pfile, int level, int reason,
 
   if (!pfile->cb.error)
     abort ();
-  rich_location richloc (src_loc);
+  rich_location richloc (pfile->line_table, src_loc);
   ret = pfile->cb.error (pfile, level, reason, &richloc, _(msgid), ap);
 
   return ret;
@@ -140,7 +140,7 @@ cpp_diagnostic_with_line (cpp_reader * pfile, int level, int reason,
   
   if (!pfile->cb.error)
     abort ();
-  rich_location richloc (src_loc);
+  rich_location richloc (pfile->line_table, src_loc);
   richloc.override_column (column);
   ret = pfile->cb.error (pfile, level, reason, &richloc, _(msgid), ap);
 
diff --git a/libcpp/include/cpplib.h b/libcpp/include/cpplib.h
index a2bdfa0..f5c2a21 100644
--- a/libcpp/include/cpplib.h
+++ b/libcpp/include/cpplib.h
@@ -237,7 +237,8 @@ struct GTY(()) cpp_identifier {
 /* A preprocessing token.  This has been carefully packed and should
    occupy 16 bytes on 32-bit hosts and 24 bytes on 64-bit hosts.  */
 struct GTY(()) cpp_token {
-  source_location src_loc;	/* Location of first char of token.  */
+  source_location src_loc;	/* Location of first char of token,
+				   together with range of full token.  */
   ENUM_BITFIELD(cpp_ttype) type : CHAR_BIT;  /* token type */
   unsigned short flags;		/* flags - see above */
 
diff --git a/libcpp/include/line-map.h b/libcpp/include/line-map.h
index c9340a6..e7608f1 100644
--- a/libcpp/include/line-map.h
+++ b/libcpp/include/line-map.h
@@ -47,7 +47,8 @@ enum lc_reason
 typedef unsigned int linenum_type;
 
 /* The typedef "source_location" is a key within the location database,
-   identifying a source location or macro expansion.
+   identifying a source location or macro expansion, along with range
+   information, and (optionally) a pointer for use by gcc.
 
    This key only has meaning in relation to a line_maps instance.  Within
    gcc there is a single line_maps instance: "line_table", declared in
@@ -69,13 +70,48 @@ typedef unsigned int linenum_type;
              |  ordmap[0]->start_location)   | first line in ordmap 0
   -----------+-------------------------------+-------------------------------
              | ordmap[1]->start_location     | First line in ordmap 1
-             | ordmap[1]->start_location+1   | First column in that line
-             | ordmap[1]->start_location+2   | 2nd column in that line
-             |                               | Subsequent lines are offset by
-             |                               | (1 << column_bits),
-             |                               | e.g. 128 for 7 bits, with a
-             |                               | column value of 0 representing
-             |                               | "the whole line".
+             | ordmap[1]->start_location+32  | First column in that line
+             |   (assuming range_bits == 5)  |
+             | ordmap[1]->start_location+64  | 2nd column in that line
+             | ordmap[1]->start_location+4096| Second line in ordmap 1
+             |   (assuming column_bits == 12)
+             |
+             |   Subsequent lines are offset by (1 << column_bits),
+             |   e.g. 4096 for 12 bits, with a column value of 0 representing
+             |   "the whole line".
+             |
+             |   Within a line, the low "range_bits" (typically 5) are used for
+             |   storing short ranges, so that there's an offset of
+             |     (1 << range_bits) between individual columns within a line,
+             |   typically 32.
+             |   The low range_bits store the offset of the end point from the
+             |   start point, and the start point is found by masking away
+             |   the range bits.
+             |
+             |   For example:
+             |      ordmap[1]->start_location+64    "2nd column in that line"
+             |   above means a caret at that location, with a range
+             |   starting and finishing at the same place (the range bits
+             |   are 0), a range of length 1.
+             |
+             |   By contrast:
+             |      ordmap[1]->start_location+68
+             |   has range bits 0x4, meaning a caret with a range starting at
+             |   that location, but with endpoint 4 columns further on: a range
+             |   of length 5.
+             |
+             |   Ranges that have caret != start, or have an endpoint too
+             |   far away to fit in range_bits are instead stored as ad-hoc
+             |   locations.  Hence for range_bits == 5 we can compactly store
+             |   tokens of length <= 32 without needing to use the ad-hoc
+             |   table.
+             |
+             |   This packing scheme means we effectively have
+             |     (column_bits - range_bits)
+             |   of bits for the columns, typically (12 - 5) = 7, for 128
+             |   columns; longer line widths are accomodated by starting a
+             |   new ordmap with a higher column_bits.
+             |
              | ordmap[2]->start_location-1   | Final location in ordmap 1
   -----------+-------------------------------+-------------------------------
              | ordmap[2]->start_location     | First line in ordmap 2
@@ -127,8 +163,101 @@ typedef unsigned int linenum_type;
   0xffffffff | UINT_MAX                      |
   -----------+-------------------------------+-------------------------------
 
-  To see how this works in practice, see the worked example in
-  libcpp/location-example.txt.  */
+   Examples of location encoding.
+
+   Packed ranges
+   =============
+
+   Consider encoding the location of a token "foo", seen underlined here
+   on line 523, within an ordinary line_map that starts at line 500:
+
+                 11111111112
+        12345678901234567890
+     522
+     523   return foo + bar;
+                  ^~~
+     524
+
+   The location's caret and start are both at line 523, column 11; the
+   location's finish is on the same line, at column 13 (an offset of 2
+   columns, for length 3).
+
+   Line 523 is offset 23 from the starting line of the ordinary line_map.
+
+   caret == start, and the offset of the finish fits within 5 bits, so
+   this can be stored as a packed range.
+
+   This is encoded as:
+      ordmap->start
+         + (line_offset << ordmap->m_column_and_range_bits)
+         + (column << ordmap->m_range_bits)
+         + (range_offset);
+   i.e. (for line offset 23, column 11, range offset 2):
+      ordmap->start
+         + (23 << 12)
+         + (11 << 5)
+         + 2;
+   i.e.:
+      ordmap->start + 0x17162
+   assuming that the line_map uses the default of 7 bits for columns and
+   5 bits for packed range (giving 12 bits for m_column_and_range_bits).
+
+
+   "Pure" locations
+   ================
+
+   These are a special case of the above, where
+      caret == start == finish
+   They are stored as packed ranges with offset == 0.
+   For example, the location of the "f" of "foo" could be stored
+   as above, but with range offset 0, giving:
+      ordmap->start
+         + (23 << 12)
+         + (11 << 5)
+         + 0;
+   i.e.:
+      ordmap->start + 0x17160
+
+
+   Unoptimized ranges
+   ==================
+
+   Consider encoding the location of the binary expression
+   below:
+
+                 11111111112
+        12345678901234567890
+     521
+     523   return foo + bar;
+                  ~~~~^~~~~
+     523
+
+   The location's caret is at the "+", line 523 column 15, but starts
+   earlier, at the "f" of "foo" at column 11.  The finish is at the "r"
+   of "bar" at column 19.
+
+   This can't be stored as a packed range since start != caret.
+   Hence it is stored as an ad-hoc location e.g. 0x80000003.
+
+   Stripping off the top bit gives us an index into the ad-hoc
+   lookaside table:
+
+     line_table->location_adhoc_data_map.data[0x3]
+
+   from which the caret, start and finish can be looked up,
+   encoded as "pure" locations:
+
+     start  == ordmap->start + (23 << 12) + (11 << 5)
+            == ordmap->start + 0x17160  (as above; the "f" of "foo")
+
+     caret  == ordmap->start + (23 << 12) + (15 << 5)
+            == ordmap->start + 0x171e0
+
+     finish == ordmap->start + (23 << 12) + (19 << 5)
+            == ordmap->start + 0x17260
+
+   To further see how source_location works in practice, see the
+   worked example in libcpp/location-example.txt.  */
 typedef unsigned int source_location;
 
 /* A range of source locations.
@@ -217,8 +346,9 @@ struct GTY((tag ("0"), desc ("%h.reason == LC_ENTER_MACRO ? 2 : 1"))) line_map {
    
    Physical source file TO_FILE at line TO_LINE at column 0 is represented
    by the logical START_LOCATION.  TO_LINE+L at column C is represented by
-   START_LOCATION+(L*(1<<column_bits))+C, as long as C<(1<<column_bits),
-   and the result_location is less than the next line_map's start_location.
+   START_LOCATION+(L*(1<<m_column_and_range_bits))+(C*1<<m_range_bits), as
+   long as C<(1<<effective range bits), and the result_location is less than
+   the next line_map's start_location.
    (The top line is line 1 and the leftmost column is column 1; line/column 0
    means "entire file/line" or "unknown line/column" or "not applicable".)
 
@@ -238,8 +368,24 @@ struct GTY((tag ("1"))) line_map_ordinary : public line_map {
      cpp_buffer.  */
   unsigned char sysp;
 
-  /* Number of the low-order source_location bits used for a column number.  */
-  unsigned int column_bits : 8;
+  /* Number of the low-order source_location bits used for column numbers
+     and ranges.  */
+  unsigned int m_column_and_range_bits : 8;
+
+  /* Number of the low-order "column" bits used for storing short ranges
+     inline, rather than in the ad-hoc table.
+     MSB                                                                 LSB
+     31                                                                    0
+     +-------------------------+-------------------------------------------+
+     |                         |<---map->column_and_range_bits (e.g. 12)-->|
+     +-------------------------+-----------------------+-------------------+
+     |                         | column_and_range_bits | map->range_bits   |
+     |                         |   - range_bits        |                   |
+     +-------------------------+-----------------------+-------------------+
+     | row bits                | effective column bits | short range bits  |
+     |                         |    (e.g. 7)           |   (e.g. 5)        |
+     +-------------------------+-----------------------+-------------------+ */
+  unsigned int m_range_bits : 8;
 };
 
 /* This is the highest possible source location encoded within an
@@ -435,15 +581,6 @@ ORDINARY_MAP_IN_SYSTEM_HEADER_P (const line_map_ordinary *ord_map)
   return ord_map->sysp;
 }
 
-/* Get the number of the low-order source_location bits used for a
-   column number within ordinary map MAP.  */
-
-inline unsigned char
-ORDINARY_MAP_NUMBER_OF_COLUMN_BITS (const line_map_ordinary *ord_map)
-{
-  return ord_map->column_bits;
-}
-
 /* Get the filename of ordinary map MAP.  */
 
 inline const char *
@@ -524,9 +661,11 @@ struct GTY(()) maps_info_macro {
   unsigned int cache;
 };
 
-/* Data structure to associate an arbitrary data to a source location.  */
+/* Data structure to associate a source_range together with an arbitrary
+   data pointer with a source location.  */
 struct GTY(()) location_adhoc_data {
   source_location locus;
+  source_range src_range;
   void * GTY((skip)) data;
 };
 
@@ -588,6 +727,12 @@ struct GTY(()) line_maps {
 
   /* True if we've seen a #line or # 44 "file" directive.  */
   bool seen_line_directive;
+
+  /* The default value of range_bits in ordinary line maps.  */
+  unsigned int default_range_bits;
+
+  unsigned int num_optimized_ranges;
+  unsigned int num_unoptimized_ranges;
 };
 
 /* Returns the number of allocated maps so far. MAP_KIND shall be TRUE
@@ -825,11 +970,15 @@ LINEMAPS_LAST_ALLOCATED_MACRO_MAP (const line_maps *set)
 
 extern void location_adhoc_data_fini (struct line_maps *);
 extern source_location get_combined_adhoc_loc (struct line_maps *,
-					       source_location, void *);
+					       source_location,
+					       source_range,
+					       void *);
 extern void *get_data_from_adhoc_loc (struct line_maps *, source_location);
 extern source_location get_location_from_adhoc_loc (struct line_maps *,
 						    source_location);
 
+extern source_range get_range_from_loc (line_maps *set, source_location loc);
+
 /* Get whether location LOC is an ad-hoc location.  */
 
 inline bool
@@ -838,14 +987,21 @@ IS_ADHOC_LOC (source_location loc)
   return (loc & MAX_SOURCE_LOCATION) != loc;
 }
 
+/* Get whether location LOC is a "pure" location, or
+   whether it is an ad-hoc location, or embeds range information.  */
+
+bool
+pure_location_p (line_maps *set, source_location loc);
+
 /* Combine LOC and BLOCK, giving a combined adhoc location.  */
 
 inline source_location
 COMBINE_LOCATION_DATA (struct line_maps *set,
 		       source_location loc,
+		       source_range src_range,
 		       void *block)
 {
-  return get_combined_adhoc_loc (set, loc, block);
+  return get_combined_adhoc_loc (set, loc, src_range, block);
 }
 
 extern void rebuild_location_adhoc_htab (struct line_maps *);
@@ -931,7 +1087,7 @@ inline linenum_type
 SOURCE_LINE (const line_map_ordinary *ord_map, source_location loc)
 {
   return ((loc - ord_map->start_location)
-	  >> ord_map->column_bits) + ord_map->to_line;
+	  >> ord_map->m_column_and_range_bits) + ord_map->to_line;
 }
 
 /* Convert a map and source_location to source column number.  */
@@ -939,7 +1095,7 @@ inline linenum_type
 SOURCE_COLUMN (const line_map_ordinary *ord_map, source_location loc)
 {
   return ((loc - ord_map->start_location)
-	  & ((1 << ord_map->column_bits) - 1));
+	  & ((1 << ord_map->m_column_and_range_bits) - 1)) >> ord_map->m_range_bits;
 }
 
 /* Return the location of the last source line within an ordinary
@@ -949,7 +1105,7 @@ LAST_SOURCE_LINE_LOCATION (const line_map_ordinary *map)
 {
   return (((map[1].start_location - 1
 	    - map->start_location)
-	   & ~((1 << map->column_bits) - 1))
+	   & ~((1 << map->m_column_and_range_bits) - 1))
 	  + map->start_location);
 }
 
@@ -999,7 +1155,8 @@ linemap_position_for_column (struct line_maps *, unsigned int);
 /* Encode and return a source location from a given line and
    column.  */
 source_location
-linemap_position_for_line_and_column (const line_map_ordinary *,
+linemap_position_for_line_and_column (line_maps *set,
+				      const line_map_ordinary *,
 				      linenum_type, unsigned int);
 
 /* Encode and return a source_location starting from location LOC and
@@ -1187,7 +1344,7 @@ class rich_location
   /* Constructors.  */
 
   /* Constructing from a location.  */
-  rich_location (source_location loc);
+  rich_location (line_maps *set, source_location loc);
 
   /* Constructing from a source_range.  */
   rich_location (source_range src_range);
diff --git a/libcpp/lex.c b/libcpp/lex.c
index 7e97bc2..d9b428a 100644
--- a/libcpp/lex.c
+++ b/libcpp/lex.c
@@ -2723,6 +2723,19 @@ _cpp_lex_direct (cpp_reader *pfile)
       break;
     }
 
+  source_range tok_range;
+  tok_range.m_start = result->src_loc;
+  if (result->src_loc >= RESERVED_LOCATION_COUNT)
+    tok_range.m_finish
+      = linemap_position_for_column (pfile->line_table,
+				     CPP_BUF_COLUMN (buffer, buffer->cur));
+  else
+    tok_range.m_finish = tok_range.m_start;
+
+  result->src_loc = COMBINE_LOCATION_DATA (pfile->line_table,
+					   result->src_loc,
+					   tok_range, NULL);
+
   return result;
 }
 
diff --git a/libcpp/line-map.c b/libcpp/line-map.c
index 3c19f93..c5aa422 100644
--- a/libcpp/line-map.c
+++ b/libcpp/line-map.c
@@ -27,9 +27,9 @@ along with this program; see the file COPYING3.  If not see
 #include "hashtab.h"
 
 /* Do not track column numbers higher than this one.  As a result, the
-   range of column_bits is [7, 18] (or 0 if column numbers are
+   range of column_bits is [12, 18] (or 0 if column numbers are
    disabled).  */
-const unsigned int LINE_MAP_MAX_COLUMN_NUMBER = (1U << 17);
+const unsigned int LINE_MAP_MAX_COLUMN_NUMBER = (1U << 12);
 
 /* Do not track column numbers if locations get higher than this.  */
 const source_location LINE_MAP_MAX_LOCATION_WITH_COLS = 0x60000000;
@@ -46,7 +46,7 @@ static const line_map_macro* linemap_macro_map_lookup (struct line_maps *,
 static source_location linemap_macro_map_loc_to_def_point
 (const line_map_macro *, source_location);
 static source_location linemap_macro_map_loc_unwind_toward_spelling
-(const line_map_macro *, source_location);
+(line_maps *set, const line_map_macro *, source_location);
 static source_location linemap_macro_map_loc_to_exp_point
 (const line_map_macro *, source_location);
 static source_location linemap_macro_loc_to_spelling_point
@@ -69,7 +69,10 @@ location_adhoc_data_hash (const void *l)
 {
   const struct location_adhoc_data *lb =
       (const struct location_adhoc_data *) l;
-  return (hashval_t) lb->locus + (size_t) lb->data;
+  return ((hashval_t) lb->locus
+	  + (hashval_t) lb->src_range.m_start
+	  + (hashval_t) lb->src_range.m_finish
+	  + (size_t) lb->data);
 }
 
 /* Compare function for location_adhoc_data hashtable.  */
@@ -81,7 +84,10 @@ location_adhoc_data_eq (const void *l1, const void *l2)
       (const struct location_adhoc_data *) l1;
   const struct location_adhoc_data *lb2 =
       (const struct location_adhoc_data *) l2;
-  return lb1->locus == lb2->locus && lb1->data == lb2->data;
+  return (lb1->locus == lb2->locus
+	  && lb1->src_range.m_start == lb2->src_range.m_start
+	  && lb1->src_range.m_finish == lb2->src_range.m_finish
+	  && lb1->data == lb2->data);
 }
 
 /* Update the hashtable when location_adhoc_data is reallocated.  */
@@ -106,23 +112,103 @@ rebuild_location_adhoc_htab (struct line_maps *set)
 		    set->location_adhoc_data_map.data + i, INSERT);
 }
 
+/* Helper function for get_combined_adhoc_loc.
+   Can the given LOCUS + SRC_RANGE and DATA pointer be stored compactly
+   within a source_location, without needing to use an ad-hoc location.  */
+
+static bool
+can_be_stored_compactly_p (struct line_maps *set,
+			   source_location locus,
+			   source_range src_range,
+			   void *data)
+{
+  /* If there's an ad-hoc pointer, we can't store it directly in the
+     source_location, we need the lookaside.  */
+  if (data)
+    return false;
+
+  /* We only store ranges that begin at the locus and that are sufficiently
+     "sane".  */
+  if (src_range.m_start != locus)
+    return false;
+
+  if (src_range.m_finish < src_range.m_start)
+    return false;
+
+  if (src_range.m_start < RESERVED_LOCATION_COUNT)
+    return false;
+
+  if (locus >= LINE_MAP_MAX_LOCATION_WITH_COLS)
+    return false;
+
+  /* All 3 locations must be within ordinary maps, typically, the same
+     ordinary map.  */
+  source_location lowest_macro_loc = LINEMAPS_MACRO_LOWEST_LOCATION (set);
+  if (locus >= lowest_macro_loc)
+    return false;
+  if (src_range.m_start >= lowest_macro_loc)
+    return false;
+  if (src_range.m_finish >= lowest_macro_loc)
+    return false;
+
+  /* Passed all tests.  */
+  return true;
+}
+
 /* Combine LOCUS and DATA to a combined adhoc loc.  */
 
 source_location
 get_combined_adhoc_loc (struct line_maps *set,
-			source_location locus, void *data)
+			source_location locus,
+			source_range src_range,
+			void *data)
 {
   struct location_adhoc_data lb;
   struct location_adhoc_data **slot;
 
-  linemap_assert (data);
-
   if (IS_ADHOC_LOC (locus))
     locus
       = set->location_adhoc_data_map.data[locus & MAX_SOURCE_LOCATION].locus;
   if (locus == 0 && data == NULL)
     return 0;
+
+  /* Any ordinary locations ought to be "pure" at this point: no
+     compressed ranges.  */
+  linemap_assert (locus < RESERVED_LOCATION_COUNT
+		  || locus >= LINE_MAP_MAX_LOCATION_WITH_COLS
+		  || locus >= LINEMAPS_MACRO_LOWEST_LOCATION (set)
+		  || pure_location_p (set, locus));
+
+  /* Consider short-range optimization.  */
+  if (can_be_stored_compactly_p (set, locus, src_range, data))
+    {
+      /* The low bits ought to be clear.  */
+      linemap_assert (pure_location_p (set, locus));
+      const line_map *map = linemap_lookup (set, locus);
+      const line_map_ordinary *ordmap = linemap_check_ordinary (map);
+      unsigned int int_diff = src_range.m_finish - src_range.m_start;
+      unsigned int col_diff = (int_diff >> ordmap->m_range_bits);
+      if (col_diff < (1U << ordmap->m_range_bits))
+	{
+	  source_location packed = locus | col_diff;
+	  set->num_optimized_ranges++;
+	  return packed;
+	}
+    }
+
+  /* We can also compactly store the reserved locations
+     when locus == start == finish (and data is NULL).  */
+  if (locus < RESERVED_LOCATION_COUNT
+      && locus == src_range.m_start
+      && locus == src_range.m_finish
+      && !data)
+    return locus;
+
+  if (!data)
+    set->num_unoptimized_ranges++;
+
   lb.locus = locus;
+  lb.src_range = src_range;
   lb.data = data;
   slot = (struct location_adhoc_data **)
       htab_find_slot (set->location_adhoc_data_map.htab, &lb, INSERT);
@@ -177,6 +263,60 @@ get_location_from_adhoc_loc (struct line_maps *set, source_location loc)
   return set->location_adhoc_data_map.data[loc & MAX_SOURCE_LOCATION].locus;
 }
 
+/* Return the source_range for adhoc location LOC.  */
+
+static source_range
+get_range_from_adhoc_loc (struct line_maps *set, source_location loc)
+{
+  linemap_assert (IS_ADHOC_LOC (loc));
+  return set->location_adhoc_data_map.data[loc & MAX_SOURCE_LOCATION].src_range;
+}
+
+/* Get the source_range of location LOC, either from the ad-hoc
+   lookaside table, or embedded inside LOC itself.  */
+
+source_range
+get_range_from_loc (struct line_maps *set,
+		    source_location loc)
+{
+  if (IS_ADHOC_LOC (loc))
+    return get_range_from_adhoc_loc (set, loc);
+
+  /* For ordinary maps, extract packed range.  */
+  if (loc >= RESERVED_LOCATION_COUNT
+      && loc < LINEMAPS_MACRO_LOWEST_LOCATION (set)
+      && loc <= LINE_MAP_MAX_LOCATION_WITH_COLS)
+    {
+      const line_map *map = linemap_lookup (set, loc);
+      const line_map_ordinary *ordmap = linemap_check_ordinary (map);
+      source_range result;
+      int offset = loc & ((1 << ordmap->m_range_bits) - 1);
+      result.m_start = loc - offset;
+      result.m_finish = result.m_start + (offset << ordmap->m_range_bits);
+      return result;
+    }
+
+  return source_range::from_location (loc);
+}
+
+/* Get whether location LOC is a "pure" location, or
+   whether it is an ad-hoc location, or embeds range information.  */
+
+bool
+pure_location_p (line_maps *set, source_location loc)
+{
+  if (IS_ADHOC_LOC (loc))
+    return false;
+
+  const line_map *map = linemap_lookup (set, loc);
+  const line_map_ordinary *ordmap = linemap_check_ordinary (map);
+
+  if (loc & ((1U << ordmap->m_range_bits) - 1))
+    return false;
+
+  return true;
+}
+
 /* Finalize the location_adhoc_data structure.  */
 void
 location_adhoc_data_fini (struct line_maps *set)
@@ -319,7 +459,19 @@ const struct line_map *
 linemap_add (struct line_maps *set, enum lc_reason reason,
 	     unsigned int sysp, const char *to_file, linenum_type to_line)
 {
-  source_location start_location = set->highest_location + 1;
+  /* Generate a start_location above the current highest_location.
+     If possible, make the low range bits be zero.  */
+  source_location start_location;
+  if (set->highest_location < LINE_MAP_MAX_LOCATION_WITH_COLS)
+    {
+      start_location = set->highest_location + (1 << set->default_range_bits);
+      if (set->default_range_bits)
+	start_location &= ~((1 << set->default_range_bits) - 1);
+      linemap_assert (0 == (start_location
+			    & ((1 << set->default_range_bits) - 1)));
+    }
+  else
+    start_location = set->highest_location + 1;
 
   linemap_assert (!(LINEMAPS_ORDINARY_USED (set)
 		    && (start_location
@@ -398,11 +550,18 @@ linemap_add (struct line_maps *set, enum lc_reason reason,
   map->to_file = to_file;
   map->to_line = to_line;
   LINEMAPS_ORDINARY_CACHE (set) = LINEMAPS_ORDINARY_USED (set) - 1;
-  map->column_bits = 0;
+  map->m_column_and_range_bits = 0;
+  map->m_range_bits = 0;
   set->highest_location = start_location;
   set->highest_line = start_location;
   set->max_column_hint = 0;
 
+  /* This assertion is placed after set->highest_location has
+     been updated, since the latter affects
+     linemap_location_from_macro_expansion_p, which ultimately affects
+     pure_location_p.  */
+  linemap_assert (pure_location_p (set, start_location));
+
   if (reason == LC_ENTER)
     {
       map->included_from =
@@ -549,13 +708,14 @@ linemap_line_start (struct line_maps *set, linenum_type to_line,
     SOURCE_LINE (map, set->highest_line);
   int line_delta = to_line - last_line;
   bool add_map = false;
+  linemap_assert (map->m_column_and_range_bits >= map->m_range_bits);
+  int effective_column_bits = map->m_column_and_range_bits - map->m_range_bits;
 
   if (line_delta < 0
       || (line_delta > 10
-	  && line_delta * ORDINARY_MAP_NUMBER_OF_COLUMN_BITS (map) > 1000)
-      || (max_column_hint >= (1U << ORDINARY_MAP_NUMBER_OF_COLUMN_BITS (map)))
-      || (max_column_hint <= 80
-	  && ORDINARY_MAP_NUMBER_OF_COLUMN_BITS (map) >= 10)
+	  && line_delta * map->m_column_and_range_bits > 1000)
+      || (max_column_hint >= (1U << effective_column_bits))
+      || (max_column_hint <= 80 && effective_column_bits >= 10)
       || (highest > LINE_MAP_MAX_LOCATION_WITH_COLS
 	  && (set->max_column_hint || highest >= LINE_MAP_MAX_SOURCE_LOCATION)))
     add_map = true;
@@ -564,22 +724,27 @@ linemap_line_start (struct line_maps *set, linenum_type to_line,
   if (add_map)
     {
       int column_bits;
+      int range_bits;
       if (max_column_hint > LINE_MAP_MAX_COLUMN_NUMBER
 	  || highest > LINE_MAP_MAX_LOCATION_WITH_COLS)
 	{
 	  /* If the column number is ridiculous or we've allocated a huge
-	     number of source_locations, give up on column numbers. */
+	     number of source_locations, give up on column numbers
+	     (and on packed ranges).  */
 	  max_column_hint = 0;
 	  column_bits = 0;
+	  range_bits = 0;
 	  if (highest > LINE_MAP_MAX_SOURCE_LOCATION)
 	    return 0;
 	}
       else
 	{
 	  column_bits = 7;
+	  range_bits = set->default_range_bits;
 	  while (max_column_hint >= (1U << column_bits))
 	    column_bits++;
 	  max_column_hint = 1U << column_bits;
+	  column_bits += range_bits;
 	}
       /* Allocate the new line_map.  However, if the current map only has a
 	 single line we can sometimes just increase its column_bits instead. */
@@ -592,14 +757,14 @@ linemap_line_start (struct line_maps *set, linenum_type to_line,
 				ORDINARY_MAP_IN_SYSTEM_HEADER_P (map),
 				ORDINARY_MAP_FILE_NAME (map),
 				to_line)));
-      map->column_bits = column_bits;
+      map->m_column_and_range_bits = column_bits;
+      map->m_range_bits = range_bits;
       r = (MAP_START_LOCATION (map)
 	   + ((to_line - ORDINARY_MAP_STARTING_LINE_NUMBER (map))
 	      << column_bits));
     }
   else
-    r = highest - SOURCE_COLUMN (map, highest)
-      + (line_delta << ORDINARY_MAP_NUMBER_OF_COLUMN_BITS (map));
+    r = set->highest_line + (line_delta << map->m_column_and_range_bits);
 
   /* Locations of ordinary tokens are always lower than locations of
      macro tokens.  */
@@ -610,6 +775,18 @@ linemap_line_start (struct line_maps *set, linenum_type to_line,
   if (r > set->highest_location)
     set->highest_location = r;
   set->max_column_hint = max_column_hint;
+
+  /* At this point, we expect one of:
+     (a) the normal case: a "pure" location with 0 range bits, or
+     (b) we've gone past LINE_MAP_MAX_LOCATION_WITH_COLS so can't track
+        columns anymore (or ranges), or
+     (c) we're in a region with a column hint exceeding
+        LINE_MAP_MAX_COLUMN_NUMBER, so column-tracking is off,
+	with column_bits == 0.  */
+  linemap_assert (pure_location_p (set, r)
+		  || r >= LINE_MAP_MAX_LOCATION_WITH_COLS
+		  || map->m_column_and_range_bits == 0);
+  linemap_assert (SOURCE_LINE (map, r) == to_line);
   return r;
 }
 
@@ -640,7 +817,8 @@ linemap_position_for_column (struct line_maps *set, unsigned int to_column)
 	  r = linemap_line_start (set, SOURCE_LINE (map, r), to_column + 50);
 	}
     }
-  r = r + to_column;
+  line_map_ordinary *map = LINEMAPS_LAST_ORDINARY_MAP (set);
+  r = r + (to_column << map->m_range_bits);
   if (r >= set->highest_location)
     set->highest_location = r;
   return r;
@@ -650,16 +828,25 @@ linemap_position_for_column (struct line_maps *set, unsigned int to_column)
    column.  */
 
 source_location
-linemap_position_for_line_and_column (const line_map_ordinary *ord_map,
+linemap_position_for_line_and_column (line_maps *set,
+				      const line_map_ordinary *ord_map,
 				      linenum_type line,
 				      unsigned column)
 {
   linemap_assert (ORDINARY_MAP_STARTING_LINE_NUMBER (ord_map) <= line);
 
-  return (MAP_START_LOCATION (ord_map)
-	  + ((line - ORDINARY_MAP_STARTING_LINE_NUMBER (ord_map))
-	     << ORDINARY_MAP_NUMBER_OF_COLUMN_BITS (ord_map))
-	  + (column & ((1 << ORDINARY_MAP_NUMBER_OF_COLUMN_BITS (ord_map)) - 1)));
+  source_location r = MAP_START_LOCATION (ord_map);
+  r += ((line - ORDINARY_MAP_STARTING_LINE_NUMBER (ord_map))
+	<< ord_map->m_column_and_range_bits);
+  if (r <= LINE_MAP_MAX_LOCATION_WITH_COLS)
+    r += ((column & ((1 << ord_map->m_column_and_range_bits) - 1))
+	  << ord_map->m_range_bits);
+  source_location upper_limit = LINEMAPS_MACRO_LOWEST_LOCATION (set);
+  if (r >= upper_limit)
+    r = upper_limit - 1;
+  if (r > set->highest_location)
+    set->highest_location = r;
+  return r;
 }
 
 /* Encode and return a source_location starting from location LOC and
@@ -673,6 +860,9 @@ linemap_position_for_loc_and_offset (struct line_maps *set,
 {
   const line_map_ordinary * map = NULL;
 
+  if (IS_ADHOC_LOC (loc))
+    loc = set->location_adhoc_data_map.data[loc & MAX_SOURCE_LOCATION].locus;
+
   /* This function does not support virtual locations yet.  */
   if (linemap_assert_fails
       (!linemap_location_from_macro_expansion_p (set, loc)))
@@ -711,11 +901,11 @@ linemap_position_for_loc_and_offset (struct line_maps *set,
     }
 
   offset += column;
-  if (linemap_assert_fails (offset < (1u << map->column_bits)))
+  if (linemap_assert_fails (offset < (1u << map->m_column_and_range_bits)))
     return loc;
 
   source_location r = 
-    linemap_position_for_line_and_column (map, line, offset);
+    linemap_position_for_line_and_column (set, map, line, offset);
   if (linemap_assert_fails (r <= set->highest_location)
       || linemap_assert_fails (map == linemap_lookup (set, r)))
     return loc;
@@ -893,14 +1083,19 @@ linemap_macro_map_loc_to_def_point (const line_map_macro *map,
    In other words, this returns the xI location presented in the
    comments of line_map_macro above.  */
 source_location
-linemap_macro_map_loc_unwind_toward_spelling (const line_map_macro* map,
+linemap_macro_map_loc_unwind_toward_spelling (line_maps *set,
+					      const line_map_macro* map,
 					      source_location location)
 {
   unsigned token_no;
 
+  if (IS_ADHOC_LOC (location))
+    location = get_location_from_adhoc_loc (set, location);
+
   linemap_assert (linemap_macro_expansion_map_p (map)
 		  && location >= MAP_START_LOCATION (map));
   linemap_assert (location >= RESERVED_LOCATION_COUNT);
+  linemap_assert (!IS_ADHOC_LOC (location));
 
   token_no = location - MAP_START_LOCATION (map);
   linemap_assert (token_no < MACRO_MAP_NUM_MACRO_TOKENS (map));
@@ -1010,7 +1205,7 @@ linemap_location_in_system_header_p (struct line_maps *set,
 
 	      /* It's a token resulting from a macro expansion.  */
 	      source_location loc =
-		linemap_macro_map_loc_unwind_toward_spelling (macro_map, location);
+		linemap_macro_map_loc_unwind_toward_spelling (set, macro_map, location);
 	      if (loc < RESERVED_LOCATION_COUNT)
 		/* This token might come from a built-in macro.  Let's
 		   look at where that macro got expanded.  */
@@ -1183,11 +1378,6 @@ linemap_macro_loc_to_spelling_point (struct line_maps *set,
 				     const line_map_ordinary **original_map)
 {
   struct line_map *map;
-
-  if (IS_ADHOC_LOC (location))
-    location = set->location_adhoc_data_map.data[location
-						 & MAX_SOURCE_LOCATION].locus;
-
   linemap_assert (set && location >= RESERVED_LOCATION_COUNT);
 
   while (true)
@@ -1198,7 +1388,7 @@ linemap_macro_loc_to_spelling_point (struct line_maps *set,
 
       location
 	= linemap_macro_map_loc_unwind_toward_spelling
-	    (linemap_check_macro (map),
+	    (set, linemap_check_macro (map),
 	     location);
     }
 
@@ -1341,10 +1531,11 @@ linemap_resolve_location (struct line_maps *set,
 			  enum location_resolution_kind lrk,
 			  const line_map_ordinary **map)
 {
+  source_location locus = loc;
   if (IS_ADHOC_LOC (loc))
-    loc = set->location_adhoc_data_map.data[loc & MAX_SOURCE_LOCATION].locus;
+    locus = set->location_adhoc_data_map.data[loc & MAX_SOURCE_LOCATION].locus;
 
-  if (loc < RESERVED_LOCATION_COUNT)
+  if (locus < RESERVED_LOCATION_COUNT)
     {
       /* A reserved location wasn't encoded in a map.  Let's return a
 	 NULL map here, just like what linemap_ordinary_map_lookup
@@ -1396,7 +1587,7 @@ linemap_unwind_toward_expansion (struct line_maps *set,
     loc = set->location_adhoc_data_map.data[loc & MAX_SOURCE_LOCATION].locus;
 
   resolved_location =
-    linemap_macro_map_loc_unwind_toward_spelling (macro_map, loc);
+    linemap_macro_map_loc_unwind_toward_spelling (set, macro_map, loc);
   resolved_map = linemap_lookup (set, resolved_location);
 
   if (!linemap_macro_expansion_map_p (resolved_map))
@@ -1478,9 +1669,9 @@ linemap_expand_location (struct line_maps *set,
   memset (&xloc, 0, sizeof (xloc));
   if (IS_ADHOC_LOC (loc))
     {
-      loc = set->location_adhoc_data_map.data[loc & MAX_SOURCE_LOCATION].locus;
       xloc.data
 	= set->location_adhoc_data_map.data[loc & MAX_SOURCE_LOCATION].data;
+      loc = set->location_adhoc_data_map.data[loc & MAX_SOURCE_LOCATION].locus;
     }
 
   if (loc < RESERVED_LOCATION_COUNT)
@@ -1760,13 +1951,14 @@ line_table_dump (FILE *stream, struct line_maps *set, unsigned int num_ordinary,
 
 /* Construct a rich_location with location LOC as its initial range.  */
 
-rich_location::rich_location (source_location loc) :
+rich_location::rich_location (line_maps *set, source_location loc) :
   m_loc (loc),
   m_num_ranges (0),
   m_have_expanded_location (false)
 {
-  /* Set up the 0th range: */
-  add_range (loc, loc, true);
+  /* Set up the 0th range, extracting any range from LOC.  */
+  source_range src_range = get_range_from_loc (set, loc);
+  add_range (src_range, true);
   m_ranges[0].m_caret = lazily_expand_location ();
 }
 
diff --git a/libcpp/location-example.txt b/libcpp/location-example.txt
index a5f95b2..14b5c2e 100644
--- a/libcpp/location-example.txt
+++ b/libcpp/location-example.txt
@@ -30,142 +30,154 @@ RESERVED LOCATIONS
   source_location interval: 0 <= loc < 2
 
 ORDINARY MAP: 0
-  source_location interval: 2 <= loc < 3
+  source_location interval: 32 <= loc < 64
   file: test.c
   starting at line: 1
-  column bits: 7
-test.c:  1|loc:    2|#include "test.h"
-                    |00000001111111111
-                    |34567890123456789
+  column bits: 12
+  range bits: 5
+test.c:  1|loc:   32|#include "test.h"
+                    |69269258258148147
+                    |46802468024680246
 
 ORDINARY MAP: 1
-  source_location interval: 3 <= loc < 4
+  source_location interval: 64 <= loc < 96
   file: <built-in>
   starting at line: 0
   column bits: 0
+  range bits: 0
 
 ORDINARY MAP: 2
-  source_location interval: 4 <= loc < 5
+  source_location interval: 96 <= loc < 128
   file: <command-line>
   starting at line: 0
   column bits: 0
+  range bits: 0
 
 ORDINARY MAP: 3
-  source_location interval: 5 <= loc < 5005
+  source_location interval: 128 <= loc < 160128
   file: /usr/include/stdc-predef.h
   starting at line: 1
-  column bits: 7
+  column bits: 12
+  range bits: 5
 (contents of /usr/include/stdc-predef.h snipped for brevity)
 
 ORDINARY MAP: 4
-  source_location interval: 5005 <= loc < 5006
+  source_location interval: 160128 <= loc < 160160
   file: <command-line>
-  starting at line: 1
-  column bits: 7
+  starting at line: 32
+  column bits: 12
+  range bits: 5
 
 ORDINARY MAP: 5
-  source_location interval: 5006 <= loc < 5134
+  source_location interval: 160160 <= loc < 164256
   file: test.c
   starting at line: 1
-  column bits: 7
-test.c:  1|loc: 5006|#include "test.h"
-                    |55555555555555555
+  column bits: 12
+  range bits: 5
+test.c:  1|loc:160160|#include "test.h"
                     |00000000000000000
-                    |00011111111112222
-                    |78901234567890123
+                    |12223334445556667
+                    |92582581481470470
+                    |24680246802468024
 
 ORDINARY MAP: 6
-  source_location interval: 5134 <= loc < 5416
+  source_location interval: 164256 <= loc < 173280
   file: test.h
   starting at line: 1
-  column bits: 7
-test.h:  1|loc: 5134|extern int foo ();
-                    |555555555555555555
-                    |111111111111111111
-                    |333334444444444555
-                    |567890123456789012
-test.h:  2|loc: 5262|
+  column bits: 12
+  range bits: 5
+test.h:  1|loc:164256|extern int foo ();
+                    |444444444444444444
+                    |233344455566677788
+                    |825814814704703603
+                    |802468024680246802
+test.h:  2|loc:168352|
                     |
                     |
                     |
                     |
-test.h:  3|loc: 5390|#define PLUS(A, B) A + B
-                    |555555555555555555555555
-                    |333333333444444444444444
-                    |999999999000000000011111
-                    |123456789012345678901234
+test.h:  3|loc:172448|#define PLUS(A, B) A + B
+                    |222222222222222223333333
+                    |455566677788889990001112
+                    |814704703603692692582581
+                    |024680246802468024680246
 
 ORDINARY MAP: 7
-  source_location interval: 5416 <= loc < 6314
+  source_location interval: 173280 <= loc < 202016
   file: test.c
   starting at line: 2
-  column bits: 7
-test.c:  2|loc: 5416|
+  column bits: 12
+  range bits: 5
+test.c:  2|loc:173280|
                     |
                     |
                     |
                     |
-test.c:  3|loc: 5544|int
-                    |555
-                    |555
+test.c:  3|loc:177376|int
+                    |777
                     |444
-                    |567
-test.c:  4|loc: 5672|main (int argc, char **argv)
-                    |5555555555555555555555555555
-                    |6666666666666666666666666667
-                    |7777777888888888899999999990
-                    |3456789012345678901234567890
-test.c:  5|loc: 5800|{
+                    |047
+                    |802
+test.c:  4|loc:181472|main (int argc, char **argv)
+                    |1111111111111111222222222222
+                    |5556666777888999000111222333
+                    |0360369269258258148147047036
+                    |4680246802468024680246802468
+test.c:  5|loc:185568|{
                     |5
-                    |8
-                    |0
-                    |1
-test.c:  6|loc: 5928|  int a = PLUS (1,2);
-                    |555555555555555555555
-                    |999999999999999999999
-                    |233333333334444444444
-                    |901234567890123456789
-test.c:  7|loc: 6056|  int b = PLUS (3,4);
-                    |666666666666666666666
-                    |000000000000000000000
-                    |555666666666677777777
-                    |789012345678901234567
-test.c:  8|loc: 6184|  return 0;
-                    |66666666666
-                    |11111111111
-                    |88888999999
-                    |56789012345
-test.c:  9|loc: 6312|}
                     |6
-                    |3
+                    |0
+                    |0
+test.c:  6|loc:189664|  int a = PLUS (1,2);
+                    |999999999900000000000
+                    |677788899900011122233
+                    |926925825814814704703
+                    |680246802468024680246
+test.c:  7|loc:193760|  int b = PLUS (3,4);
+                    |333333344444444444444
+                    |788899900011122233344
+                    |925825814814704703603
+                    |246802468024680246802
+test.c:  8|loc:197856|  return 0;
+                    |77778888888
+                    |89990001112
+                    |82581481470
+                    |80246802468
+test.c:  9|loc:201952|}
                     |1
-                    |3
+                    |9
+                    |8
+                    |4
 
 UNALLOCATED LOCATIONS
-  source_location interval: 6314 <= loc < 2147483633
+  source_location interval: 202016 <= loc < 2147483633
 
 MACRO 1: PLUS (7 tokens)
   source_location interval: 2147483633 <= loc < 2147483640
-test.c:7:11: note: expansion point is location 6067
+test.c:7:11: note: expansion point is location 194115
    int b = PLUS (3,4);
-           ^
+           ^~~~
+
   map->start_location: 2147483633
   macro_locations:
-    0: 6073, 5410
-test.c:7:17: note: token 0 has x-location == 6073
+    0: 194304, 173088
+test.c:7:17: note: token 0 has x-location == 194304
    int b = PLUS (3,4);
                  ^
-test.c:7:17: note: token 0 has y-location == 5410
-    1: 5412, 5412
+
+test.c:7:17: note: token 0 has y-location == 173088
+    1: 173152, 173152
 In file included from test.c:1:0:
-test.h:3:22: note: token 1 has x-location == y-location == 5412
+test.h:3:22: note: token 1 has x-location == y-location == 173152
  #define PLUS(A, B) A + B
                       ^
-    2: 6075, 5414
-test.c:7:19: note: token 2 has x-location == 6075
+
+    2: 194368, 173216
+test.c:7:19: note: token 2 has x-location == 194368
    int b = PLUS (3,4);
                    ^
-test.c:7:19: note: token 2 has y-location == 5414
+
+test.c:7:19: note: token 2 has y-location == 173216
     3: 0, 2947526575
 cc1: note: token 3 has x-location == 0
 cc1: note: token 3 has y-location == 2947526575
@@ -178,26 +190,30 @@ x-location == y-location == 2947526575 encodes token # 800042942
 
 MACRO 0: PLUS (7 tokens)
   source_location interval: 2147483640 <= loc < 2147483647
-test.c:6:11: note: expansion point is location 5939
+test.c:6:11: note: expansion point is location 190019
    int a = PLUS (1,2);
-           ^
+           ^~~~
+
   map->start_location: 2147483640
   macro_locations:
-    0: 5945, 5410
-test.c:6:17: note: token 0 has x-location == 5945
+    0: 190208, 173088
+test.c:6:17: note: token 0 has x-location == 190208
    int a = PLUS (1,2);
                  ^
-test.c:6:17: note: token 0 has y-location == 5410
-    1: 5412, 5412
+
+test.c:6:17: note: token 0 has y-location == 173088
+    1: 173152, 173152
 In file included from test.c:1:0:
-test.h:3:22: note: token 1 has x-location == y-location == 5412
+test.h:3:22: note: token 1 has x-location == y-location == 173152
  #define PLUS(A, B) A + B
                       ^
-    2: 5947, 5414
-test.c:6:19: note: token 2 has x-location == 5947
+
+    2: 190272, 173216
+test.c:6:19: note: token 2 has x-location == 190272
    int a = PLUS (1,2);
                    ^
-test.c:6:19: note: token 2 has y-location == 5414
+
+test.c:6:19: note: token 2 has y-location == 173216
     3: 0, 2947526575
 cc1: note: token 3 has x-location == 0
 cc1: note: token 3 has y-location == 2947526575
-- 
1.8.5.3

Reply via email to