Re: [Dev] API Manager - Retrieving JSON Payload During Mediation

[Amitha Dissanayake]

Hi Lahiru,

Thanks for the clarification. It cleared my doubt.
Amitha
On Sep 27 2018, at 8:02 am, Lahiru Madushanka  wrote:
>
> Hi Amith,
>
> Explanation of how json-eval() works.
>
> We have two kinds of path expressions in synapse, synapseJsonPath and 
> synapseXPath
> Following method is where we evaluate the json-eval() expressions
>
> public String stringValueOf(MessageContext synCtx) {
> org.apache.axis2.context.MessageContext amc = ((Axis2MessageContext) 
> synCtx).getAxis2MessageContext();
> InputStream stream;
> if (!JsonUtil.hasAJsonPayload(amc) || "true".equals(enableStreamingJsonPath)) 
> {
> try {
> if (null == amc.getEnvelope().getBody().getFirstElement()) {
> // Get message from PT Pipe.
> stream = getMessageInputStreamPT(amc);
> if (stream == null) {
> stream = JsonUtil.getJsonPayload(amc);
> } else {
> JsonUtil.getNewJsonPayload(amc, stream, true, true);
> }
> } else {
> // Message Already built.
> stream = JsonUtil.toJsonStream(amc.getEnvelope().getBody().getFirstElement());
> }
> if(stream != null) {
> return stringValueOf(stream);
> }else{
> log.warn("Json Payload is empty.");
> return "";
> }
> } catch (IOException e) {
> handleException("Could not find JSON Stream in PassThrough Pipe during JSON 
> path evaluation.", e);
> }
> } else {
> stream = JsonUtil.getJsonPayload(amc);
> return stringValueOf(stream);
> }
> return "";
> }
>
>
> In all the branches we use stringValueOf(stream) method to evaluate the 
> jsonPath expressions against an input-stream.
> Where we take the stream is dependent on certain conditions as in the above 
> implementation.
>
> If message already build we convert message body to a JSON stream in
> > stream = 
> > JsonUtil.toJsonStream(amc.getEnvelope().getBody().getFirstElement());
> >
> >
>
> If message is not already built we take the stream from passthrough Pipe
>
>
> > Pipe pipe= (Pipe) 
> > context.getProperty(PassThroughConstants.PASS_THROUGH_PIPE);
> >
>
> In the final else condition we take the stream from a property in axis2 
> message context
>
>
> > Object o = 
> > messageContext.getProperty(ORG_APACHE_SYNAPSE_COMMONS_JSON_JSON_INPUT_STREAM);
> >
>
>
> You can find all implementation details in [1]
>
> [1] https://github.com/wso2/wso2-synapse
>
> Thanks,
> Lahiru
>
>
>
>
>
>
>
> On Wed, Sep 26, 2018 at 7:31 PM Nuwan Dias  (mailto:nuw...@wso2.com)> wrote:
> > Adding Isuru and Shafreen.
> >
> > The mediation engine (EI/API Gateway) supports transforming or changing 
> > messages from various protocols to various other protocols. Such as from 
> > XML to JSON, from Text to XML, from form-url to JSON, likewise. To support 
> > these various types of transformations, upon receiving a message the engine 
> > would first convert it into a common base, and then convert it to the 
> > requested base, likewise. Without doing which, the combinations of these 
> > transformations would be endless. In the case of EI/API Gateway, this 
> > common base is XML. So every message, if you need to process it, is first 
> > converted to XML and then all changes or transformation are done on the 
> > particular XML message and then converted to whatever the requested format 
> > is. So as far as I know you can't work with the message on plain JSON 
> > format only. However, I've added two of my colleagues who have a better 
> > idea about this than me to see if this is possible.
> >
> > Thanks,
> > NuwanD.
> >
> >
> > On Wed, Sep 26, 2018 at 2:11 PM Amitha Dissanayake  > (mailto:ami...@yaalalabs.com)> wrote:
> > > Hi Nuwan,
> > >
> > > Thanks a lot for the prompt response. That document helped. However, my 
> > > original question still remains unresolved to me.
> > > More specifically, when using the class mediator, I used the property 
> > > mediator and received the JSON string to the class. But I still have to 
> > > convert to XML and load it to the axis2 context message envelope (i.e. 
> > > SOAP).
> > > 
> > > 
> > >
> > >
> > >
> > > I couldn't find how this json-eval() function evaluates the JSON payload 
> > > (Whether it's accessing a JSON representation or an XML representation).
> > > How can I do this without converting the JSON to XML (without converting 
> > > it to XML and attaching to axis2 message)?
> > > How is JSON represented in the EI / API Manager by default (XML or JSON)? 
> > > I understand that conversions and manipulations are possible either way.
> > > Can the the EI / API Manager keep the JSON payload not converted 
> > > end-to-end (i.e. from the point of backend REST API sending the response 
> > > with a JSON payload till the requester receiving it?
> > > I'm evaluating API Manager, while hoping to apply it to a latency and 
> > > performance critical application. That is why I'm trying to understand 
> > > how it works in the EI.
> > > Thanks.
> > > Regards,
> > > Amitha
> > >
> > > On Sep 26 2018, at 2:12 pm, Nuwan Dias  > > (mailto:nuw...@wso2.com)> wrote:
> > > >
> > > > Hi Amitha,
> > > >
> > > > Maybe this document [1] would help 

Re: [Dev] API Manager - Retrieving JSON Payload During Mediation

[Amitha Dissanayake]

Hi Nuwan,

Thanks a lot. I'm worried about the latency overhead these conversions add as I 
would be working with large payloads.
@Isuru, Shafreen, Would you let me know/share resources, if there's a way to 
bypass these conversions.
@All, Is there an initiative to to migrate from XML and intermediate SOAP based 
message handling in future versions as JSON is faster in parsing and XML is 
slow and cumbersome?
Thanks.
Regards,
Amitha

On Sep 26 2018, at 7:29 pm, Nuwan Dias  wrote:
>
> Adding Isuru and Shafreen.
>
> The mediation engine (EI/API Gateway) supports transforming or changing 
> messages from various protocols to various other protocols. Such as from XML 
> to JSON, from Text to XML, from form-url to JSON, likewise. To support these 
> various types of transformations, upon receiving a message the engine would 
> first convert it into a common base, and then convert it to the requested 
> base, likewise. Without doing which, the combinations of these 
> transformations would be endless. In the case of EI/API Gateway, this common 
> base is XML. So every message, if you need to process it, is first converted 
> to XML and then all changes or transformation are done on the particular XML 
> message and then converted to whatever the requested format is. So as far as 
> I know you can't work with the message on plain JSON format only. However, 
> I've added two of my colleagues who have a better idea about this than me to 
> see if this is possible.
>
> Thanks,
> NuwanD.
>
>
> On Wed, Sep 26, 2018 at 2:11 PM Amitha Dissanayake  (mailto:ami...@yaalalabs.com)> wrote:
> > Hi Nuwan,
> >
> > Thanks a lot for the prompt response. That document helped. However, my 
> > original question still remains unresolved to me.
> > More specifically, when using the class mediator, I used the property 
> > mediator and received the JSON string to the class. But I still have to 
> > convert to XML and load it to the axis2 context message envelope (i.e. 
> > SOAP).
> > 
> > 
>
>
>

___
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev

Re: [Dev] API Manager - Retrieving JSON Payload During Mediation

[Lahiru Madushanka]

Hi Amith,

Explanation of how json-eval() works.

We have two kinds of path expressions in synapse, synapseJsonPath and
synapseXPath
Following method is where we evaluate the json-eval() expressions

public String stringValueOf(MessageContext synCtx) {
org.apache.axis2.context.MessageContext amc =
((Axis2MessageContext) synCtx).getAxis2MessageContext();
InputStream stream;
if (!JsonUtil.hasAJsonPayload(amc) ||
"true".equals(enableStreamingJsonPath)) {
try {
if (null == amc.getEnvelope().getBody().getFirstElement()) {
// Get message from PT Pipe.
stream = getMessageInputStreamPT(amc);
if (stream == null) {
stream = JsonUtil.getJsonPayload(amc);
} else {
JsonUtil.getNewJsonPayload(amc, stream, true, true);
}
} else {
// Message Already built.
stream =
JsonUtil.toJsonStream(amc.getEnvelope().getBody().getFirstElement());
}
if(stream != null) {
return stringValueOf(stream);
}else{
log.warn("Json Payload is empty.");
return "";
}
} catch (IOException e) {
handleException("Could not find JSON Stream in PassThrough
Pipe during JSON path evaluation.", e);
}
} else {
stream = JsonUtil.getJsonPayload(amc);
return stringValueOf(stream);
}
return "";
}


In all the branches we use *stringValueOf(stream)* method to evaluate the
jsonPath expressions against an input-stream.
Where we take the stream is dependent on certain conditions as in the above
implementation.

If message already build we convert message body to a JSON stream in

stream =
JsonUtil.toJsonStream(amc.getEnvelope().getBody().getFirstElement());

If message is not already built we take the stream from passthrough Pipe

Pipe pipe= (Pipe)
context.getProperty(PassThroughConstants.PASS_THROUGH_PIPE);

In the final else condition we take the stream from a property in axis2
message context

Object o =
messageContext.getProperty(ORG_APACHE_SYNAPSE_COMMONS_JSON_JSON_INPUT_STREAM);


You can find all implementation details in [1]

[1] https://github.com/wso2/wso2-synapse

Thanks,
Lahiru

On Wed, Sep 26, 2018 at 7:31 PM Nuwan Dias  wrote:

> Adding Isuru and Shafreen.
>
> The mediation engine (EI/API Gateway) supports transforming or changing
> messages from various protocols to various other protocols. Such as from
> XML to JSON, from Text to XML, from form-url to JSON, likewise. To support
> these various types of transformations, upon receiving a message the engine
> would first convert it into a common base, and then convert it to the
> requested base, likewise. Without doing which, the combinations of these
> transformations would be endless. In the case of EI/API Gateway, this
> common base is XML. So every message, if you need to process it, is first
> converted to XML and then all changes or transformation are done on the
> particular XML message and then converted to whatever the requested format
> is. So as far as I know you can't work with the message on plain JSON
> format only. However, I've added two of my colleagues who have a better
> idea about this than me to see if this is possible.
>
> Thanks,
> NuwanD.
>
> On Wed, Sep 26, 2018 at 2:11 PM Amitha Dissanayake 
> wrote:
>
>> Hi Nuwan,
>>
>> Thanks a lot for the prompt response. That document helped. However, my
>> original question still remains unresolved to me.
>>
>> More specifically, when using the class mediator, I used the property
>> mediator and received the JSON string to the class. But I still have to
>> convert to XML and load it to the axis2 context message envelope (i.e.
>> SOAP).
>>
>> 
>> 
>>
>>
>>
>>
>> I couldn't find how this json-eval() function evaluates the JSON payload
>> (Whether it's accessing a JSON representation or an XML representation).
>>
>> How can I do this without converting the JSON to XML (without converting
>> it to XML and attaching to axis2 message)?
>>
>> How is JSON represented in the EI / API Manager by default (XML or JSON)?
>> I understand that conversions and manipulations are possible either way.
>>
>> Can the the EI / API Manager keep the JSON payload not converted
>> end-to-end (i.e. from the point of backend REST API sending the response
>> with a JSON payload till the requester receiving it?
>>
>> I'm evaluating API Manager, while hoping to apply it to a latency and
>> performance critical application. That is why I'm trying to understand how
>> it works in the EI.
>>
>> Thanks.
>>
>> Regards,
>> Amitha
>>
>> On Sep 26 2018, at 2:12 pm, Nuwan Dias  wrote:
>>
>>
>> Hi Amitha,
>>
>> Maybe this document [1] would help you understand how to deal with JSON
>> payloads better.
>>
>> [1] -
>> https://docs.wso2.com/display/EI630/Working+with+JSON+Message+Payloads
>>
>> Thanks,
>> NuwanD.
>>
>> On Wed, Sep 26, 2018 

Re: [Dev] API Manager - Retrieving JSON Payload During Mediation

[Nuwan Dias]

Hi Amitha,

Maybe this document [1] would help you understand how to deal with JSON
payloads better.

[1] - https://docs.wso2.com/display/EI630/Working+with+JSON+Message+Payloads

Thanks,
NuwanD.

On Wed, Sep 26, 2018 at 10:01 AM Amitha Dissanayake 
wrote:

> Hi,
>
> I'm a developer who's exploring API Manager. When I was attempting to
> retrieve the JSON payload form a Context Message during mediation (out
> sequence), I observed that the payload it attached in a SOAP Envelope
> inside the Axis2 Context Message. However, I'm receiving the response from
> a REST endpoint.
>
> In that case I had to convert the SOAP Evelope XML to JSON, then modify
> the changes and again convert it to XML and attach to Axis2 Context
> Message. In that operation, 4 conversions happen. That is,
>
>1. Between the endpoint and mediation point. JSON to XML
>2. During mediation, I convert XML to JSON and do the operation,
>3. I again convert the JSON to XML and attach to the axis2 message.
>4. Postman receives a JSON; means XML has again been converted inside
>the API Manager.
>
>
> When I ran the mediation on debug mode, I observed that the content is
> received as SOAPAction to the mediation point.
>
> I feel that this adds an unnecessary latency overhead or I am doing
> something wrong in the process. Is there a way to overcome this issue/
> directly access the JSON payload or what is the correct way to access the
> JSON payload without too many conversions?
>
> Thanks in Advance.
>
> Regards,
>
> Amitha Dissanayake
> Senior Software Engineer [image: Logo] 
>
> Yaala Labs
> 14 Sir Baron Jayathilake Mawatha
> Colombo 1, Sri Lanka
> m: + 94 77 548 1350
> e: ami...@yaalalabs.com
>
> www.yaalalabs.com [image: LinkedIn icon]
> 
> ___
> Dev mailing list
> Dev@wso2.org
> http://wso2.org/cgi-bin/mailman/listinfo/dev
>


-- 
*Nuwan Dias* | Director | WSO2 Inc.
(m) +94 777 775 729 | (e) nuw...@wso2.com
[image: Signature.jpg]
___
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev

[Dev] Open pull requests for charon

[Lösch , Sebastian]

Hi Devs,



there is a bunch of open pull requests on github for the Charon project:

https://github.com/wso2/charon/pulls

Is anybody out there having time to do some reviews? This would be great!



Thanks in advance!

Sebastian

___
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev

[Dev] API Manager - Retrieving JSON Payload During Mediation

[Amitha Dissanayake]

Hi,

I'm a developer who's exploring API Manager. When I was attempting to retrieve 
the JSON payload form a Context Message during mediation (out sequence), I 
observed that the payload it attached in a SOAP Envelope inside the Axis2 
Context Message. However, I'm receiving the response from a REST endpoint.
In that case I had to convert the SOAP Evelope XML to JSON, then modify the 
changes and again convert it to XML and attach to Axis2 Context Message. In 
that operation, 4 conversions happen. That is,
Between the endpoint and mediation point. JSON to XML

During mediation, I convert XML to JSON and do the operation,

I again convert the JSON to XML and attach to the axis2 message.

Postman receives a JSON; means XML has again been converted inside the API 
Manager.

When I ran the mediation on debug mode, I observed that the content is received 
as SOAPAction to the mediation point.
I feel that this adds an unnecessary latency overhead or I am doing something 
wrong in the process. Is there a way to overcome this issue/ directly access 
the JSON payload or what is the correct way to access the JSON payload without 
too many conversions?
Thanks in Advance.
Regards,
Amitha Dissanayake
Senior Software Engineer

Yaala Labs

14 Sir Baron Jayathilake Mawatha

Colombo 1, Sri Lanka

m: + 94 77 548 1350

e: ami...@yaalalabs.com (mailto:ami...@yaalalabs.com)

www.yaalalabs.com (http://www.yaalalabs.com/)

___
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev

[Dev] [IS] InCommon Federation Compliance for WSO2IS - UI Component

[Sahan Gunathilaka]

Hi All,
One of the major requirements to comply WSO2 IS with InCommon Federation is
to support *auto refreshment* of metadata of participating Service
Providers and Identity Providers at IS. Following is the designed UI to
provide this requirement.

[image: Screenshot_2018-09-26 WSO2 Management Console.png]

*"Enable Auto Refresh*" check box and "*Refresh Time*" drop box can be set
to automate the refreshing cycle. All service providers' and identity
providers' configurations will be updated according to them after clicking
on "*Save*" button.

"*Force Refresh*" button lets users to refresh metadata instantly based on
the latest update of metadata file at the InCommon Federation.

"*Clear All*" button will remove all the configured data and result "0" for
both "Total Service Providers" and "Total Identity Providers" in the table.

If you have any suggestion on this, please let me know.

Thank you!
Best Regards!

-- 
*Sahan Gunathilaka*
Intern - Software Engineering
*WSO2*
mobile:  +94776343266

[image: http://wso2.com/signature]
___
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev

[Dev] Regarding Carbon Crypto Service usages

[Mevan Karunanayake]

Hi all,

Currently I have developed an Carbon Crypto API implementation to use
HSM(Hardware Security Module) as a cryptographic provider. So in order to
test the component, I need to find usages of the Carbon Crypto API. I was
only able to find a usage of InternalCryptoProvider to encrypt/decrypt
tenant keystore usernames and passwords.

I am very grateful if anyone can clarify me whether there is any other
usages of the Carbon Crypto API other than the above usage.

Carbon Crypto Service - https://github.com/wso2/carbon-crypto-service/
Usage of InternalCryptoProvider -
https://github.com/wso2/carbon-kernel/blob/4.4.x/core/org.wso2.carbon.core/src/main/java/org/wso2/carbon/core/util/CryptoUtil.java

Regards,

*Mevan Karunanayake*
Trainee Software Engineer | WSO2
Email : me...@wso2.com
Mobile : +94 71 202 8954
___
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev