Very likely this problem is not related to nested documents but to
fielddata loading because of the "integer" field. Field data is a
column-oriented view of the data that is, by default, lazily loaded from
the inverted index on the first time that it is needed, and then cached
until the end of life
I have an index that uses 1 level of nested documents. When I run a query
on it the result comes back in about 20-200 milliseconds. When I add a
facet or an aggregation involving the nested documents the uncached
response always takes 2-3 seconds, regardless of how many documents have
been sele