Hi NuwanD,
I have found the root cause for this issue. As mentioned in [1], this error
is coming from kernel level and that has been propagated to the synapse
level. Actual root cause is mentioned below.
[2015-10-07 17:13:38,135] ERROR - ServerWorker Error processing GET reguest
for : /t/api.com/responseapi/weather?q=London. Error detail: null.
java.util.EmptyStackException
at java.util.Stack.peek(Stack.java:102)
at java.util.Stack.pop(Stack.java:84)
at
org.wso2.carbon.context.internal.CarbonContextDataHolder.endTenantFlow(CarbonContextDataHolder.java:1291)
at
org.wso2.carbon.context.PrivilegedCarbonContext.endTenantFlow(PrivilegedCarbonContext.java:75)
at
org.wso2.carbon.core.multitenancy.MultitenantMessageReceiver.processRequest(MultitenantMessageReceiver.java:229)
at
org.wso2.carbon.core.multitenancy.MultitenantMessageReceiver.receive(MultitenantMessageReceiver.java:78)
at org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:180)
at
org.apache.synapse.transport.passthru.ServerWorker.processNonEntityEnclosingRESTHandler(ServerWorker.java:317)
at
org.apache.synapse.transport.passthru.ServerWorker.run(ServerWorker.java:149)
at
org.apache.axis2.transport.base.threads.NativeWorkerPool$1.run(NativeWorkerPool.java:172)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
There is a separate mail thread to discuss about this issue and previously
Jagath has given a fix but that was not merged to the kernel. I will reply
to that thread and continue the fix on that thread.
[1] https://wso2.org/jira/browse/ESBJAVA-4142
Thanks,
Chanaka
On Tue, Oct 6, 2015 at 4:49 PM, Chanaka Fernando <chana...@wso2.com> wrote:
> Hi NuwanD,
>
> I can reproduce the issue with the provided configuration. I have already
> created a fix for this issue and currently testing it. I'll update the
> thread once I finalized the fix and merge the PR.
>
> On Tue, Oct 6, 2015 at 2:59 PM, Nuwan Dias <nuw...@wso2.com> wrote:
>
>> Attaching the two APIs I used for testing. To reproduce this issue,
>> create a tenant with the domain as 'wso2.com' and deploy these two APIs
>> within the tenant space and restart the server.
>>
>> Then access the API without the version using the following URL.
>>
>> http://localhost:8280/t/wso2.com/responseapi/weather?q=London
>>
>> Thanks,
>> NuwanD.
>>
>> On Tue, Oct 6, 2015 at 2:53 PM, Chanaka Fernando <chana...@wso2.com>
>> wrote:
>>
>>> Hi Nuwan,
>>>
>>> I had a look into the code which is causing this NPE. According to the
>>> way it is written, this NPE can be occurred. I have tried to replicate what
>>> NuwanD has mentioned but couldn't reproduce the same NPE. This can be
>>> occurred due to synapse configuration issue.
>>>
>>> @NuwanD: Can you share the configurations which you have used to
>>> reproduce this issue?
>>>
>>>
>>> Thanks,
>>> Chanaka
>>>
>>> On Thu, Oct 1, 2015 at 10:41 AM, Chanaka Fernando <chana...@wso2.com>
>>> wrote:
>>>
>>>> Hi All,
>>>>
>>>> We are looking into this issue.
>>>>
>>>> On Thu, Oct 1, 2015 at 10:37 AM, Sandapa Handakumbura <sand...@wso2.com
>>>> > wrote:
>>>>
>>>>> Hi Nuwan,
>>>>>
>>>>> I faced a similar issue during ESB 4.9.0 release testing. First
>>>>> request for proxy services (after tenant unloading/ server restart) failed
>>>>> in tenant mode [1]. Unfortunately ESB team could not reproduce it and I
>>>>> could not identify the exact steps to reproduce it either.
>>>>>
>>>>> Hope the team will look in to this since you have identified steps to
>>>>> reproduce the issue.
>>>>>
>>>>> [1] https://wso2.org/jira/browse/ESBJAVA-4142
>>>>>
>>>>> Thanks,
>>>>> Sandapa
>>>>>
>>>>>
>>>>> On Thu, Oct 1, 2015 at 4:47 AM, Nuwan Dias <nuw...@wso2.com> wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> I have a scenario on the ESB where I have created two APIs with the
>>>>>> same context in the tenant space. The first API doesn't have a version
>>>>>> and
>>>>>> the second API has a version. The first API receives the request and
>>>>>> forwards to the second API. See illustration below.
>>>>>>
>>>>>> Client (/twitter) --> Twitter_API (/twitter/1.0.0) -->
>>>>>> Twitter_API_v1.0.0
>>>>>>
>>>>>> This scenario however causes the following [1] NPE on the first
>>>>>> request that's sent to the server. The subsequent requests pass until a
>>>>>> server restart happens.
>>>>>>
>>>>>> I tried logging into the management console of the ESB using tenant
>>>>>> credentials to check whether it was related to tenant loading. The first
>>>>>> request still failed with the same error. Any idea why this is happening?
>>>>>> This issue first occurred to me in API Manager after migrating to the
>>>>>> latest released version of Synapse. And is causing test failures in the
>>>>>> product. It worked fine in the older versions of Synapse (2.1.2- version
>>>>>> range).
>>>>>>
>>>>>> [1]
>>>>>>
>>>>>> [2015-10-01 04:41:39,519] ERROR - PassThroughHttpSender Failed to
>>>>>> submit the response
>>>>>> java.lang.NullPointerException
>>>>>> at
>>>>>> org.apache.synapse.transport.passthru.util.SourceResponseFactory.create(SourceResponseFactory.java:64)
>>>>>> at
>>>>>> org.apache.synapse.transport.passthru.PassThroughHttpSender.submitResponse(PassThroughHttpSender.java:462)
>>>>>> at
>>>>>> org.apache.synapse.transport.passthru.PassThroughHttpSender.invoke(PassThroughHttpSender.java:267)
>>>>>> at org.apache.axis2.engine.AxisEngine.send(AxisEngine.java:442)
>>>>>> at
>>>>>> org.wso2.carbon.core.multitenancy.transports.TenantTransportSender.invoke(TenantTransportSender.java:231)
>>>>>> at org.apache.axis2.engine.AxisEngine.send(AxisEngine.java:442)
>>>>>> at
>>>>>> org.apache.synapse.core.axis2.Axis2Sender.sendBack(Axis2Sender.java:212)
>>>>>> at
>>>>>> org.apache.synapse.core.axis2.Axis2SynapseEnvironment.send(Axis2SynapseEnvironment.java:444)
>>>>>> at
>>>>>> org.apache.synapse.mediators.builtin.SendMediator.mediate(SendMediator.java:102)
>>>>>> at
>>>>>> org.apache.synapse.mediators.AbstractListMediator.mediate(AbstractListMediator.java:81)
>>>>>> at
>>>>>> org.apache.synapse.mediators.AbstractListMediator.mediate(AbstractListMediator.java:48)
>>>>>> at
>>>>>> org.apache.synapse.mediators.base.SequenceMediator.mediate(SequenceMediator.java:149)
>>>>>> at org.apache.synapse.rest.Resource.process(Resource.java:297)
>>>>>> at org.apache.synapse.rest.API.process(API.java:335)
>>>>>> at
>>>>>> org.apache.synapse.rest.RESTRequestHandler.dispatchToAPI(RESTRequestHandler.java:97)
>>>>>> at
>>>>>> org.apache.synapse.rest.RESTRequestHandler.process(RESTRequestHandler.java:52)
>>>>>> at
>>>>>> org.apache.synapse.core.axis2.Axis2SynapseEnvironment.injectMessage(Axis2SynapseEnvironment.java:266)
>>>>>> at
>>>>>> org.apache.synapse.core.axis2.SynapseCallbackReceiver.handleMessage(SynapseCallbackReceiver.java:529)
>>>>>> at
>>>>>> org.apache.synapse.core.axis2.SynapseCallbackReceiver.receive(SynapseCallbackReceiver.java:172)
>>>>>> at org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:180)
>>>>>> at
>>>>>> org.wso2.carbon.core.multitenancy.MultitenantMessageReceiver.processResponse(MultitenantMessageReceiver.java:166)
>>>>>> at
>>>>>> org.wso2.carbon.core.multitenancy.MultitenantMessageReceiver.receive(MultitenantMessageReceiver.java:82)
>>>>>> at org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:180)
>>>>>> at
>>>>>> org.apache.synapse.transport.passthru.ClientWorker.run(ClientWorker.java:247)
>>>>>> at
>>>>>> org.apache.axis2.transport.base.threads.NativeWorkerPool$1.run(NativeWorkerPool.java:172)
>>>>>> at
>>>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>>>>>> at
>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>>>>>> at java.lang.Thread.run(Thread.java:745)
>>>>>> [2015-10-01 04:41:39,521] ERROR - Axis2Sender
>>>>>> Access-Control-Allow-Credentials:true,Access-Control-Allow-Methods:GET,
>>>>>> POST,Access-Control-Allow-Origin:*,Content-Type:application/json;
>>>>>> charset=utf-8,X-Source:redis,<?xml version='1.0'
>>>>>> encoding='utf-8'?><soapenv:Envelope xmlns:soapenv="
>>>>>> http://www.w3.org/2003/05/soap-envelope";><soapenv:Body/></soapenv:Envelope>
>>>>>> Unexpected error sending message back
>>>>>> org.apache.axis2.AxisFault: Failed to submit the response
>>>>>> at
>>>>>> org.apache.synapse.transport.passthru.PassThroughHttpSender.handleException(PassThroughHttpSender.java:610)
>>>>>> at
>>>>>> org.apache.synapse.transport.passthru.PassThroughHttpSender.invoke(PassThroughHttpSender.java:269)
>>>>>> at org.apache.axis2.engine.AxisEngine.send(AxisEngine.java:442)
>>>>>> at
>>>>>> org.wso2.carbon.core.multitenancy.transports.TenantTransportSender.invoke(TenantTransportSender.java:231)
>>>>>> at org.apache.axis2.engine.AxisEngine.send(AxisEngine.java:442)
>>>>>> at
>>>>>> org.apache.synapse.core.axis2.Axis2Sender.sendBack(Axis2Sender.java:212)
>>>>>> at
>>>>>> org.apache.synapse.core.axis2.Axis2SynapseEnvironment.send(Axis2SynapseEnvironment.java:444)
>>>>>> at
>>>>>> org.apache.synapse.mediators.builtin.SendMediator.mediate(SendMediator.java:102)
>>>>>> at
>>>>>> org.apache.synapse.mediators.AbstractListMediator.mediate(AbstractListMediator.java:81)
>>>>>> at
>>>>>> org.apache.synapse.mediators.AbstractListMediator.mediate(AbstractListMediator.java:48)
>>>>>> at
>>>>>> org.apache.synapse.mediators.base.SequenceMediator.mediate(SequenceMediator.java:149)
>>>>>> at org.apache.synapse.rest.Resource.process(Resource.java:297)
>>>>>> at org.apache.synapse.rest.API.process(API.java:335)
>>>>>> at
>>>>>> org.apache.synapse.rest.RESTRequestHandler.dispatchToAPI(RESTRequestHandler.java:97)
>>>>>> at
>>>>>> org.apache.synapse.rest.RESTRequestHandler.process(RESTRequestHandler.java:52)
>>>>>> at
>>>>>> org.apache.synapse.core.axis2.Axis2SynapseEnvironment.injectMessage(Axis2SynapseEnvironment.java:266)
>>>>>> at
>>>>>> org.apache.synapse.core.axis2.SynapseCallbackReceiver.handleMessage(SynapseCallbackReceiver.java:529)
>>>>>> at
>>>>>> org.apache.synapse.core.axis2.SynapseCallbackReceiver.receive(SynapseCallbackReceiver.java:172)
>>>>>> at org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:180)
>>>>>> at
>>>>>> org.wso2.carbon.core.multitenancy.MultitenantMessageReceiver.processResponse(MultitenantMessageReceiver.java:166)
>>>>>> at
>>>>>> org.wso2.carbon.core.multitenancy.MultitenantMessageReceiver.receive(MultitenantMessageReceiver.java:82)
>>>>>> at org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:180)
>>>>>> at
>>>>>> org.apache.synapse.transport.passthru.ClientWorker.run(ClientWorker.java:247)
>>>>>> at
>>>>>> org.apache.axis2.transport.base.threads.NativeWorkerPool$1.run(NativeWorkerPool.java:172)
>>>>>> at
>>>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>>>>>> at
>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>>>>>> at java.lang.Thread.run(Thread.java:745)
>>>>>> Caused by: java.lang.NullPointerException
>>>>>> at
>>>>>> org.apache.synapse.transport.passthru.util.SourceResponseFactory.create(SourceResponseFactory.java:64)
>>>>>> at
>>>>>> org.apache.synapse.transport.passthru.PassThroughHttpSender.submitResponse(PassThroughHttpSender.java:462)
>>>>>> at
>>>>>> org.apache.synapse.transport.passthru.PassThroughHttpSender.invoke(PassThroughHttpSender.java:267)
>>>>>> ... 25 more
>>>>>>
>>>>>> Thanks,
>>>>>> NuwanD.
>>>>>>
>>>>>> --
>>>>>> Nuwan Dias
>>>>>>
>>>>>> Technical Lead - WSO2, Inc. http://wso2.com
>>>>>> email : nuw...@wso2.com
>>>>>> Phone : +94 777 775 729
>>>>>>
>>>>>> _______________________________________________
>>>>>> Dev mailing list
>>>>>> Dev@wso2.org
>>>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Sandapa Handakumbura
>>>>> Associate Technical Lead
>>>>> WSO2.Inc. ; http://wso2.com/
>>>>> M : +94777115866
>>>>>
>>>>> _______________________________________________
>>>>> Dev mailing list
>>>>> Dev@wso2.org
>>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> --
>>>> Chanaka Fernando
>>>> Senior Technical Lead
>>>> WSO2, Inc.; http://wso2.com
>>>> lean.enterprise.middleware
>>>>
>>>> mobile: +94 773337238
>>>> Blog : http://soatutorials.blogspot.com
>>>> LinkedIn:http://www.linkedin.com/pub/chanaka-fernando/19/a20/5b0
>>>> Twitter:https://twitter.com/chanakaudaya
>>>>
>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> --
>>> Chanaka Fernando
>>> Senior Technical Lead
>>> WSO2, Inc.; http://wso2.com
>>> lean.enterprise.middleware
>>>
>>> mobile: +94 773337238
>>> Blog : http://soatutorials.blogspot.com
>>> LinkedIn:http://www.linkedin.com/pub/chanaka-fernando/19/a20/5b0
>>> Twitter:https://twitter.com/chanakaudaya
>>>
>>>
>>>
>>>
>>>
>>
>>
>> --
>> Nuwan Dias
>>
>> Technical Lead - WSO2, Inc. http://wso2.com
>> email : nuw...@wso2.com
>> Phone : +94 777 775 729
>>
>
>
>
> --
> --
> Chanaka Fernando
> Senior Technical Lead
> WSO2, Inc.; http://wso2.com
> lean.enterprise.middleware
>
> mobile: +94 773337238
> Blog : http://soatutorials.blogspot.com
> LinkedIn:http://www.linkedin.com/pub/chanaka-fernando/19/a20/5b0
> Twitter:https://twitter.com/chanakaudaya
>
>
>
>
>
--
--
Chanaka Fernando
Senior Technical Lead
WSO2, Inc.; http://wso2.com
lean.enterprise.middleware
mobile: +94 773337238
Blog : http://soatutorials.blogspot.com
LinkedIn:http://www.linkedin.com/pub/chanaka-fernando/19/a20/5b0
Twitter:https://twitter.com/chanakaudaya
_______________________________________________
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev
