Hi Isuru,
Here we are trying to avoid building the message in gateway itself
since building
the entire malicious payload may lead gateway crashes. Instead of counting
limits after building the tree, We manually build the message through
third-party library and parser will keep counting nodes while parsing
object. So when the limit hit, it will terminate the process and stop
further processing. We have planned to thwart following attacks[1].Here we
provide both depth limits and schema validations.
[1]
XML Bomb
Coercive Parsing
XXE (External Entity Attacks)
Schema Poisoning
Buffer Overflows
Regards
Hasunie
On Tue, Jan 2, 2018 at 8:22 PM, Isuru Udana <isu...@wso2.com> wrote:
> Hi Hasunie,
>
> Are we gaining a significant performance improvement in this approach by
> making mediators content-unaware ?
> To perform these validations messages should be built and we are
> internally using woodstox in our builders too.
>
> At mediator level generally we shouldn't mess-up with input-streams,
> buffers and these various internal properties in Passthrough Transport.
> If it is inevitable, we need to be extra careful as these are very
> sensitive.
>
> Thanks.
>
> On Tue, Jan 2, 2018 at 5:22 PM, Hasunie Adikari <hasu...@wso2.com> wrote:
>
>> I'm trying to combine SQL injection(Regex) threat protector with the XML
>> threat protector. So I created a sequence[1] with XMLthreatprotector
>> mediator and regex mediator consecutively and uploaded it to be able to
>> validate the request message through both the xml validator and regex
>> validators. If I set the *message.builder.invoked *property to *TRUE *in xml
>> validator
>> mediator to avoid sending the content in pass-through pipe(request
>> message) as the response, Regex mediators is getting failed. The regex
>> mediator was designed a way that the incoming messages are built in synapse
>> level and eveluate the message content at the mediator level. It seems
>> like we can't continue any mediators which are required to get the message
>> content, after we manually set the aforementioned property to true in the
>> previous mediator. If I set it to true, RelayUtill will skip building the
>> message as in here [2]. Any thoughts regarding the issue. I'm currently
>> working on the issue to be able to combine regex and XML threat protectors.
>> [2]
>>
>> if (pipe != null
>> && !Boolean.TRUE.equals(messageContext
>>
>> .getProperty(PassThroughConstants.MESSAGE_BUILDER_INVOKED)) && forcePTBuild)
>> {
>> InputStream in = pipe.getInputStream();
>>
>> Object http_sc = messageContext.getProperty(NhttpConstants.HTTP_SC);
>> if (http_sc != null && http_sc instanceof Integer &&
>> http_sc.equals(202)) {
>>
>> messageContext.setProperty(PassThroughConstants.MESSAGE_BUILDER_INVOKED,
>> Boolean.TRUE);
>> return;
>> }
>>
>> builldMessage(messageContext, earlyBuild, in);
>> return;
>> }
>>
>>
>>
>> [1]
>> <sequence xmlns="http://ws.apache.org/ns/synapse"; name="validator">
>> <property name="threatType" expression="get-property('threatType')"
>> value="SQL-Injection and XML validator"/>
>> <property>
>> <property>
>> --------
>> <switch source="get-property('To')">
>> --------------
>> </switch>
>> *<class
>> name="org.wso2.carbon.apimgt.gateway.mediators.XMLSchemaValidator"/>*
>> <property name="enabledCheckPathParams"
>> expression="get-property('enabledCheckPathParams')"
>> value="true"/>
>> <property>
>> <property>
>> --
>> *<class
>> name="org.wso2.carbon.apimgt.gateway.mediators.RegularExpressionProtector"/>*
>> <filter source="get-property('threat')" regex=".*error.*">
>> <then>
>> <sequence key="_threat_fault_"></sequence>
>> </then>
>> </filter>
>> </sequence>
>>
>> Regards
>> Hasunie
>>
>> On Fri, Dec 22, 2017 at 5:14 PM, Hasunie Adikari <hasu...@wso2.com>
>> wrote:
>>
>>> Hi all,
>>>
>>> I'm working on threat protector feature in APIM. We're actually trying
>>> to achieve here is to protect both backend resources and gateway from
>>> the XML and JSON based attacks. The Balerina based APIM 3 gateway will be
>>> protected by threat handlers. But In here
>>> APIM 2.1.x we have implemented mediators to achieve it.
>>>
>>> If we allow building the request message at the synapse level, It will
>>> definitely affect the gateway, All the request messages which go through
>>> the mediators are built since the Abstarctemediator is designed a way that
>>> the isContentAware method always returns true. So we set it to false in
>>> both XML and JSON validator mediators and allow to parse the XML request
>>> via a third party StAX parser called woodstox hence it was the best option
>>> among other StAX parsers for threat protection features. It will keep
>>> counting the given limits and when the limit is exceeded, It will terminate
>>> the process and throw a meaningful exception. I have created a custom
>>> threat sequence(thrat_fault) and If a threat is detected by getting an
>>> exception I configured to direct the response through the custom error
>>> sequence.
>>> I reuse the same custom sequence which was implemented for the regex
>>> threat protector [1]
>>>
>>> Woodstox parser covers most of the vulnerabilities as in here
>>>
>>> *Vulnerablity:*
>>>
>>> *xml bomb* - DTD disabling
>>>
>>> *external entity attack* - disabling external entities.
>>>
>>> Note :
>>> Apart from the mediator level, The external entity
>>> reference property was disabled from the DOM parsers at the synapse level
>>> as well.
>>>
>>> import org.apache.xerces.impl.Constants;
>>>
>>> private static final int ENTITY_EXPANSION_LIMIT = 0;
>>> private static final DocumentBuilderFactory documentBuilderFactory =
>>>
>>> DocumentBuilderFactory.newInstance();
>>>
>>> static {
>>> documentBuilderFactory.setNamespaceAware(true);
>>> documentBuilderFactory.setXIncludeAware(false);
>>> documentBuilderFactory.setExpandEntityReferences(false);
>>>
>>> try {
>>> documentBuilderFactory.setFeature(Constants.SAX_FEATURE_PREFIX +
>>> Constants.EXTERNAL_GENERAL_ENTITIES_FEATURE, false);
>>> } catch (ParserConfigurationException e) {
>>>
>>> *Buffer overflow attack* - by limiting the count of elements, children
>>> and length of attributes/keys/values.
>>>
>>> *woodstox properties:*
>>> dtdEnabled
>>> externalEntitiesEnabled
>>> maxDepth
>>> maxElementCount
>>> maxAttributeCount
>>> maxAttributeLength
>>> entityExpansionLimit
>>> maxChildrenPerElement
>>>
>>>
>>> For thwart cohesive attacks, we use both schema validator and depth
>>> limits. Ideally, only the woodstox validator should detect the cohesive
>>> attacks by exceeding the defined depth limit. But the schema validator will
>>> protect the schema poising attacks in the second step as well.
>>>
>>> I observed an issue when It comes to combining each other(woodstox+
>>> schema validator). We have designed the feature in such a way that gets the
>>> inputstream from the message context and consumes it in the woodstox
>>> validator. but in here we have to consume the input stream again for the
>>> schema validation just after passing through the woodstox.That was the
>>> issue and I tried the following methodologies to resolved the issue
>>>
>>> 1. try to get the XML object from the woodstox parser to be able to
>>> avoid using the input stream again.
>>> 2. deep clone the inputstream and use cloned input stream for the schema
>>> validation.
>>> 3. reset, mark the buffered input stream(synapse engine also has done
>>> rest, mark)
>>>
>>> 1st one was taken time and much complex to get the XML object since
>>> Woodstock is based on the StAX parsers and also deep cloning was not
>>> working properly and experienced the same issue after cloning the
>>> inputstream. But the 3rd option makes life easy so I implemented a way that
>>> returning the buffered input stream, after doing the rest, mark, then It
>>> works properly. I went through the RelayUtil message builders [2] and It
>>> also uses the mark and reset methodology and return InputStream.
>>>
>>>
>>> I observed another issue once the validator throws an exception, the
>>> server hanged and didn't get any response and getting timeout issue. I was
>>> able to figure it out and Issue occurred while trying to build the request
>>> message in Relayutil.buildmessage().But Ideally, If we get an error we
>>> don't need the request message anymore. As I discussed offline with the
>>> APIM team, I used the *consumeAndDiscardMessage* method to discard the
>>> request message from the message context and set *message.builder.invoked
>>> *property to *TRUE. *It needs to be set to avoid sending the content in
>>> pass-through pipe (request message) as the response.
>>>
>>>
>>> [1] https://docs.wso2.com/display/AM210/Regular+Expression+T
>>> hreat+Protection+for+API+Gateway
>>> [2] https://github.com/wso2/wso2-synapse/blob/master/modules
>>> /transports/core/nhttp/src/main/java/org/apache/synapse/tran
>>> sport/passthru/util/RelayUtils.java#L121
>>>
>>>
>>> Regards
>>> Hasunie
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> --
>>> *Hasunie Adikari*
>>> Senior Software Engineer
>>> WSO2 Inc.; http://wso2.com
>>> lean.enterprise.middleware
>>> blog http://hasuniea.blogspot.com | https://medium.com/@Hasunie/
>>> Mobile:+94713095876 <+94%2071%20309%205876>
>>>
>>>
>>
>>
>> --
>> *Hasunie Adikari*
>> Senior Software Engineer
>> WSO2 Inc.; http://wso2.com
>> lean.enterprise.middleware
>> blog http://hasuniea.blogspot.com | https://medium.com/@Hasunie/
>> Mobile:+94713095876 <071%20309%205876>
>>
>>
>
>
> --
> *Isuru Udana*
> Senior Technical Lead
> WSO2 Inc.; http://wso2.com
> email: isu...@wso2.com cell: +94 77 3791887 <+94%2077%20379%201887>
> blog: http://mytecheye.blogspot.com/
>
--
*Hasunie Adikari*
Senior Software Engineer
WSO2 Inc.; http://wso2.com
lean.enterprise.middleware
blog http://hasuniea.blogspot.com | https://medium.com/@Hasunie/
Mobile:+94713095876
_______________________________________________
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
