Re: arbitrary memory allocation

[Jeff King]

On Thu, Nov 26, 2015 at 05:06:35AM +0100, ytr...@sdf-eu.org wrote:

> First, something I still don t understand, should I always ulimit ram
> usage for security purposes when I m manage a public server?

You didn't define "public" here. For serving fetches, the memory tends
to be fairly bounded and dependent on the repo you're serving. For
accepting pushes, it's trivial to convince the server to allocate a lot
of memory (you can send an unbounded set of ref updates, or you can
simply send a 50GB object that compresses down to a tiny size).

Git does not have any internal memory controls, and will generally rely
on malloc() to tell it when it is not being reasonable. I'd suggest
using OS-level memory controls like cgroups if you're hosting something
public.

-Peff
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: arbitrary memory allocation

[Junio C Hamano]

ytr...@sdf-eu.org writes:

> line_list="0032want "+obj[1][:40]+'\n'
> while len(line_list)<65430: # Get the ideal tcp packet size for fastest 
> bandwidth (64Ko)
>   for i in obj:
>   if (i==obj[0]) or (i==obj[1]) or ("pull" in i):
>   continue
>   line_list+="0032want "+i[:40]+'\n'
>   if len(line_list)>65480:
>   break
> ...
> line_list_len=line_list.count('\n')*56 # Line lengths of the pkt-line format 
> won t fill the ram, so remove them from the size counter
> count=line_list_len
> while True:
>   sys.stdout.flush()
>   sockobj.send(line_list) # for each line, the git-send-pack process 
> allocate append a member to a struct objects array
>   print("\r%.2f Mo of ram filled" % float(count/float(1048576))),
>   count+=line_list_len

This seems to be attempting to throw "want " that are
outside the original server-side advertisement over and over.  Even
though the set of distinct "want" lines you can throw at the server
is bounded by the server-side advertisement (i.e. usually you won't
be able to throw an object name that does not appear at the tip), by
repeating the requests, you seem to be hoping that you can exhaust
the object_array() used in upload-pack.c::receive_needs().

But does that attack actually work?  After seeing these "want"
lines, the object name read from there goes through this code:

o = parse_object(sha1_buf);
if (!o)
die("git upload-pack: not our ref %s",
sha1_to_hex(sha1_buf));
if (!(o->flags & WANTED)) {
o->flags |= WANTED;
if (!is_our_ref(o))
has_non_tip = 1;
add_object_array(o, NULL, _obj);
}

So it appears to me that the requests the code makes in the second
and subsequent iterations of "while True:" loop would merely be an
expensive no-op, without bloating memory footprint.

It does waste CPU cycle and network socket, though.

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: arbitrary memory allocation

[Stefan Beller]

On Mon, Nov 30, 2015 at 4:17 PM, Junio C Hamano  wrote:
> ytr...@sdf-eu.org writes:
>
>> line_list="0032want "+obj[1][:40]+'\n'
>> while len(line_list)<65430: # Get the ideal tcp packet size for fastest 
>> bandwidth (64Ko)
>>   for i in obj:
>>   if (i==obj[0]) or (i==obj[1]) or ("pull" in i):
>>   continue
>>   line_list+="0032want "+i[:40]+'\n'
>>   if len(line_list)>65480:
>>   break
>> ...
>> line_list_len=line_list.count('\n')*56 # Line lengths of the pkt-line format 
>> won t fill the ram, so remove them from the size counter
>> count=line_list_len
>> while True:
>>   sys.stdout.flush()
>>   sockobj.send(line_list) # for each line, the git-send-pack process 
>> allocate append a member to a struct objects array
>>   print("\r%.2f Mo of ram filled" % float(count/float(1048576))),
>>   count+=line_list_len
>
> This seems to be attempting to throw "want " that are
> outside the original server-side advertisement over and over.  Even
> though the set of distinct "want" lines you can throw at the server
> is bounded by the server-side advertisement (i.e. usually you won't
> be able to throw an object name that does not appear at the tip), by
> repeating the requests, you seem to be hoping that you can exhaust
> the object_array() used in upload-pack.c::receive_needs().
>
> But does that attack actually work?  After seeing these "want"
> lines, the object name read from there goes through this code:
>
> o = parse_object(sha1_buf);
> if (!o)
> die("git upload-pack: not our ref %s",
> sha1_to_hex(sha1_buf));
> if (!(o->flags & WANTED)) {
> o->flags |= WANTED;
> if (!is_our_ref(o))
> has_non_tip = 1;
> add_object_array(o, NULL, _obj);

(Looking quickly), I do not see a deduplication in add_object_array,
so you could send the same want line again and again,
to inflate the want_obj array.

If you happen to know a large object in a well known project
(some linux blob maybe?), it would be held a lots of times in memory,
which may trigger the OOM killer in linux?

> }
>
> So it appears to me that the requests the code makes in the second
> and subsequent iterations of "while True:" loop would merely be an
> expensive no-op, without bloating memory footprint.
>
> It does waste CPU cycle and network socket, though.
>
> --
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majord...@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html