[
https://issues.apache.org/jira/browse/AVRO-1111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Percy updated AVRO-1111:
-----------------------------
Attachment: AVRO-1111-1.patch
I think this patch is more limited but it's a first pass. What it does it try
to calculate whether an OOM will likely result from the listSize allocation
from the RPC header field, and if so throw an AvroRemoteException. It also
closes the socket.
I think it makes more sense to set an arbitrary limit on the listSize and check
for that since we should throw at closer to 1000 instead of like 7 billion or
something. But I'm not familiar enough with the Responder implementation to
know whether something like that is a bad idea or not... would there be
repercussions to that?
There are also other places where we should do some validation, such as the
length headers before the byte buffers... but that is a little tricky too
(arbitrary limits can be problematic for obvious reasons). I'd like some
suggestions for how to handle this as the current state leaves the protocol
pretty brittle.
> Malformed data can cause OutOfMemoryError in Avro IPC
> -----------------------------------------------------
>
> Key: AVRO-1111
> URL: https://issues.apache.org/jira/browse/AVRO-1111
> Project: Avro
> Issue Type: Bug
> Components: java
> Affects Versions: 1.6.3
> Reporter: Hari Shreedharan
> Attachments: AVRO-1111-1.patch
>
>
> If the data that comes in through the Netty channel buffer is not framed
> correctly/is not valid Avro data, then the incoming data can cause
> arbitrarily large array lists to be created, causing OutOfMemoryError.
> The relevant code(org.apache.avro.ipc.NettyTransportCodec):
> private boolean decodePackHeader(ChannelHandlerContext ctx, Channel channel,
> ChannelBuffer buffer) throws Exception {
> if (buffer.readableBytes()<8) { return false; }
> int serial = buffer.readInt();
> listSize = buffer.readInt();
> dataPack = new NettyDataPack(serial, new ArrayList<ByteBuffer>(listSize));
> return true;
> }
> If the buffer does not have valid Avro data, the listSize variable can have
> arbitrary values, causing massive ArrayLists to be created, leading to
> OutOfMemoryErrors.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
