[ https://issues.apache.org/jira/browse/HBASE-14799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15003329#comment-15003329 ]
Andrew Purtell edited comment on HBASE-14799 at 11/13/15 1:11 AM: ------------------------------------------------------------------ I investigated the test failures and found some issues. The first is we never added efficient support for serializing our Pair type. We rely on generic object serialization for it. I fixed this problem. Unfortunately I cannot be 100% backwards compatible. We can't just whitelist Pair. A Pair can hold any other type of object. We get to see that we have a Pair, but not the types contained within until after deserialization, and that's too late. Therefore I've added a code for Pair and special case handling for it, like we do with List. Older peers will not understand this change. The APIs affected are HMasterInterface#getAlterStatus and HRegionInterface#bulkLoadHFiles. Sorry, cannot be helped and avoid risk of exploit. However, thankfully it's only two APIs that are not super commonly used. To reiterate, 100% compatibility won't be possible. If that is required, then we must close this as Wont Fix. I also discovered we are generically serializing the java.lang.* types in some cases. However we will handle the primitive types in a backwards compatible way if we simply unbox, so I do this where we can. Newer peers will be able to communicate with older peers without issue. If older peers elect to send object-serialized primitives, though, newer peers will reject the message unless configured to accept legacy serialization. This is intended behavior. I'm still working through 0.94 tests. was (Author: apurtell): I investigated the test failures and found some issues. The first is we never added efficient support for serializing our Pair type. We rely on generic object serialization for it. I fixed this problem. Unfortunately I cannot be 100% backwards compatible. We can't just whitelist Pair. A Pair can hold any other type of object. We get to see that we have a Pair, but not the types contained within until after deserialization, and that's too late. Therefore I've added a code for Pair and special case handling for it, like we do with List. Older peers will not understand this change. The APIs affected are HMasterInterface#getAlterStatus and HRegionInterface#bulkLoadHFiles. Sorry, cannot be helped and avoid risk of exploit. However, thankfully it's only two APIs that are not super commonly used. To reiterate, 100% compatibility won't be possible. If that is required, then we must close this as Wont Fix. I also discovered we are generically serializing the java.lang.* types in some cases. However we will handle the primitive types in a backwards compatible way if we simply unbox, so I do this where we can. Newer peers will be able to communicate with older peers without issue. If older peers elect send object-serialized primitives, though, newer peers will reject the message unless configured to accept legacy serialization. This is intended behavior. I'm still working through 0.94 tests. > Commons-collections object deserialization remote command execution > vulnerability > ---------------------------------------------------------------------------------- > > Key: HBASE-14799 > URL: https://issues.apache.org/jira/browse/HBASE-14799 > Project: HBase > Issue Type: Bug > Reporter: Andrew Purtell > Assignee: Andrew Purtell > Priority: Critical > Fix For: 0.94.28, 0.98.17 > > Attachments: HBASE-14799-0.94.patch, HBASE-14799-0.98.patch > > > Read: > http://foxglovesecurity.com/2015/11/06/what-do-weblogic-websphere-jboss-jenkins-opennms-and-your-application-have-in-common-this-vulnerability/ > TL;DR: If you have commons-collections on your classpath and accept and > process Java object serialization data, then you probably have an exploitable > remote command execution vulnerability. > 0.94 and earlier HBase releases are vulnerable because we might read in and > rehydrate serialized Java objects out of RPC packet data in > HbaseObjectWritable using ObjectInputStream#readObject (see > https://hbase.apache.org/0.94/xref/org/apache/hadoop/hbase/io/HbaseObjectWritable.html#714) > and we have commons-collections on the classpath on the server. > 0.98 also carries some limited exposure to this problem through inclusion of > backwards compatible deserialization code in > HbaseObjectWritableFor96Migration. This is used by the 0.94-to-0.98 migration > utility, and by the AccessController when reading permissions from the ACL > table serialized in legacy format by 0.94. Unprivileged users cannot run the > tool nor access the ACL table. > Unprivileged users can however attack a 0.94 installation. An attacker might > be able to use the method discussed on that blog post to capture valid HBase > RPC payloads for 0.94 and prior versions, rewrite them to embed an exploit, > and replay them to trigger a remote command execution with the privileges of > the account under which the HBase RegionServer daemon is running. > We need to make a patch release of 0.94 that changes HbaseObjectWritable to > disallow processing of random Java object serializations. This will be a > compatibility break that might affect old style coprocessors, which quite > possibly may rely on this catch-all in HbaseObjectWritable for custom object > (de)serialization. We can introduce a new configuration setting, > "hbase.allow.legacy.object.serialization", defaulting to false. > To be thorough, we can also use the new configuration setting > "hbase.allow.legacy.object.serialization" (defaulting to false) in 0.98 to > prevent the AccessController from falling back to the vulnerable legacy code. > This turns out to not affect the ability to migrate permissions because > TablePermission implements Writable, which is safe, not Serializable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)