Thanks Chamila,
I will go through it and get back to you.
Best regards
Alapan Sur
Project Manager - SD
ValueLabs - 'inspired by potential'
Desk: +91-40-6623-9000 | Ext: 32112
Mobile: +91-7799681717
This e-mail may contain information that is proprietary, confidential or
otherwise protected from disclosure and is sent for the intended
recipient(s) only. For email disclaimer details, please visit:
http://www.valuelabs.com/terms-of-use/
From: Chamila Adhikarinayake [mailto:chami...@wso2.com]
Sent: Friday, April 27, 2018 2:39 PM
To: alapan....@valuelabs.com
Cc: Abimaran Kugathasan <abima...@wso2.com>; dev <dev@wso2.org>
Subject: Re: [Dev] One Specific API not responding
Hi Alapan,
For SQL error 1, can you try out the the solution suggested in
stackoverflow[1]
For SQL error 2, we have already fixed this in APIM Analytics 2.2.0. Issue
reported for this can be found in here[2] and the fix can be found in
here[3]
[1] https://stackoverflow.com/a/42787873/1110305
[2] https://wso2.org/jira/browse/ANLYAPIM-154
[3] https://github.com/wso2/analytics-apim/pull/369/files
Chamila.
On Wed, Apr 25, 2018 at 11:31 AM, <alapan....@valuelabs.com
<mailto:alapan....@valuelabs.com> > wrote:
Also Abi & Nuwan,
I am getting below SQL errors every minute :
SQL Error 1:
<PostgreSQL JDBC Driver, apimuser, wso2_analytics_processed_data_store_db,
10.138.120.215, 2018-04-23 12:03:57 CDT>STATEMENT: INSERT INTO
ANX___7LtOYcOI_ (partition_key, timestamp, data, record_id) VALUES ($1, $2,
$3, $4)
<PostgreSQL JDBC Driver, apimuser, wso2_analytics_processed_data_store_db,
10.138.120.215, 2018-04-23 12:03:57 CDT>ERROR: duplicate key value violates
unique constraint "anx___7ltoycoi__pkey"
<PostgreSQL JDBC Driver, apimuser, wso2_analytics_processed_data_store_db,
10.138.120.215, 2018-04-23 12:03:57 CDT>DETAIL: Key
(record_id)=(1fc48692-f672-3b1a-be77-2583bfd05bfa) already exists.
<PostgreSQL JDBC Driver, apimuser, wso2_analytics_processed_data_store_db,
10.138.120.215, 2018-04-23 12:03:57 CDT>STATEMENT: INSERT INTO
ANX___7LtOYcOI_ (partition_key, timestamp, data, record_id) VALUES ($1, $2,
$3, $4)
<PostgreSQL JDBC Driver, apimuser, wso2_analytics_processed_data_store_db,
10.138.120.215, 2018-04-23 12:03:57 CDT>ERROR: duplicate key value violates
unique constraint "anx___7ltoycoi__pkey"
<PostgreSQL JDBC Driver, apimuser, wso2_analytics_processed_data_store_db,
10.138.120.215, 2018-04-23 12:03:57 CDT>DETAIL: Key
(record_id)=(1da80896-bc33-34b2-92f1-2665cbc8c11f) already exists.
<PostgreSQL JDBC Driver, apimuser, wso2_analytics_processed_data_store_db,
10.138.120.215, 2018-04-23 12:03:57 CDT>STATEMENT: INSERT INTO
ANX___7LtOYcOI_ (partition_key, timestamp, data, record_id) VALUES ($1, $2,
$3, $4)
<PostgreSQL JDBC Driver, apimuser, wso2_analytics_processed_data_store_db,
10.138.120.215, 2018-04-23 12:03:57 CDT>ERROR: duplicate key value violates
unique constraint "anx___7ltoycoi__pkey"
<PostgreSQL JDBC Driver, apimuser, wso2_analytics_processed_data_store_db,
10.138.120.215, 2018-04-23 12:03:57 CDT>DETAIL: Key
(record_id)=(1da80896-bc33-34b2-92f1-2665cbc8c11f) already exists.
SQL Error 2:
Driver stacktrace:
at
org.apache.spark.sql.jdbc.carbon.package$CarbonJDBCWrite$.saveTable(carbon.scala:78)
at
org.apache.spark.sql.jdbc.carbon.CarbonJDBCRelation.insert(JDBCRelation.scala:263)
... 26 more
Caused by: org.apache.spark.SparkException: Job aborted due to stage
failure: Task 0 in stage 3793.0 failed 1 times, most recent failure: Lost
task 0.0 in stage 3793.0 (TID 6627, localhost):
java.sql.BatchUpdateException: Batch entry 3 INSERT INTO
API_REQ_USER_BROW_SUMMARY (api, version, apiPublisher, tenantDomain,
total_request_count, year, month, day, requestTime, os, browser) VALUES
('Hardware-Products', 'v1', 'synnex', 'carbon.super', 4, 2018, 4, 20,
1524189658000, NULL, NULL) ON CONFLICT
(api,version,apiPublisher,year,month,day,os,browser,tenantDomain) DO UPDATE
SET total_request_count=EXCLUDED.total_request_count,
requestTime=EXCLUDED.requestTime was aborted: ERROR: null value in column
"os" violates not-null constraint
Detail: Failing row contains (Hardware-Products, v1, synnex, carbon.super,
4, 2018, 4, 20, 1524189658000, null, null). Call getNextException to see
other errors in the batch.
at
org.postgresql.jdbc.BatchResultHandler.handleError(BatchResultHandler.java:148)
at
org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2191)
at
org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:472)
at org.postgresql.jdbc.PgStatement.executeBatch(PgStatement.java:791)
at
org.postgresql.jdbc.PgPreparedStatement.executeBatch(PgPreparedStatement.java:1563)
at
org.apache.spark.sql.jdbc.carbon.package$CarbonJDBCWrite$.savePartition(carbon.scala:149)
at
org.apache.spark.sql.jdbc.carbon.package$CarbonJDBCWrite$$anonfun$saveTable$1.apply(carbon.scala:72)
at
org.apache.spark.sql.jdbc.carbon.package$CarbonJDBCWrite$$anonfun$saveTable$1.apply(carbon.scala:71)
at
org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1$$anonfun$apply$33.apply(RDD.scala:920)
at org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1$$anonfun
Best regards
Alapan Sur
Project Manager - SD
ValueLabs - 'inspired by potential'
Desk: +91-40-6623-9000 | Ext: 32112
Mobile: +91-7799681717
This e-mail may contain information that is proprietary, confidential or
otherwise protected from disclosure and is sent for the intended
recipient(s) only. For email disclaimer details, please visit:
http://www.valuelabs.com/terms-of-use/
From: alapan....@valuelabs.com <mailto:alapan....@valuelabs.com>
[mailto:alapan....@valuelabs.com <mailto:alapan....@valuelabs.com> ]
Sent: Wednesday, April 25, 2018 10:01 AM
To: Abimaran Kugathasan <abima...@wso2.com <mailto:abima...@wso2.com> >
Cc: Nuwan Dias <nuw...@wso2.com <mailto:nuw...@wso2.com> >; dev
<dev@wso2.org <mailto:dev@wso2.org> >
Subject: RE: [Dev] One Specific API not responding
Yes Abi. It hits and gives the response back in couple of seconds.
Best regards
Alapan Sur
Project Manager - SD
ValueLabs - 'inspired by potential'
Desk: +91-40-6623-9000 | Ext: 32112
Mobile: +91-7799681717
This e-mail may contain information that is proprietary, confidential or
otherwise protected from disclosure and is sent for the intended
recipient(s) only. For email disclaimer details, please visit:
http://www.valuelabs.com/terms-of-use/
From: Abimaran Kugathasan [mailto:abima...@wso2.com]
Sent: Wednesday, April 25, 2018 9:36 AM
To: alapan....@valuelabs.com <mailto:alapan....@valuelabs.com>
Cc: Nuwan Dias <nuw...@wso2.com <mailto:nuw...@wso2.com> >; dev
<dev@wso2.org <mailto:dev@wso2.org> >
Subject: Re: [Dev] One Specific API not responding
Hi Alapan,
When you get Gateway timeout errors, do you see the request hit to your
backend services?
On Tue, Apr 24, 2018 at 6:10 PM, <alapan....@valuelabs.com
<mailto:alapan....@valuelabs.com> > wrote:
Thanks Nuwan, we will try that.
Just to give you few more data pointers:
1. We are getting around 100 Kb of data as response.
2. When we do restart of the application everything work like charm
but after few 5 – 6 hours or 10 K – 15 K API hits, we are started getting
Gateway timeout errors.
Best regards
Alapan Sur
Project Manager - SD
ValueLabs - 'inspired by potential'
Desk: +91-40-6623-9000 | Ext: 32112
Mobile: +91-7799681717
This e-mail may contain information that is proprietary, confidential or
otherwise protected from disclosure and is sent for the intended
recipient(s) only. For email disclaimer details, please visit:
http://www.valuelabs.com/terms-of-use/
From: Nuwan Dias [mailto:nuw...@wso2.com <mailto:nuw...@wso2.com> ]
Sent: Tuesday, April 24, 2018 5:55 PM
To: alapan....@valuelabs.com <mailto:alapan....@valuelabs.com>
Cc: Abimaran Kugathasan <abima...@wso2.com <mailto:abima...@wso2.com> >; dev
<dev@wso2.org <mailto:dev@wso2.org> >
Subject: Re: [Dev] One Specific API not responding
I was specifically interested in seeing the value of the Transfer-Encoding
request header. Since it appears to be missing, can you try disabling
chunking of this particular API by setting <property name="DISABLE_CHUNKING"
value="true" scope="axis2"/> in the api xml file located at
<APIM_HOME>/repository/deployment/server/synapse-configs/default/api ? You
should put this config within the <inSequence> of the <resource> being
executed.
If the above works, you could apply this solution as documented at [1].
[1] - https://docs.wso2.com/display/AM210/Disable+Message+Chunking
Thanks,
Nuwan.
On Tue, Apr 24, 2018 at 5:47 PM, <alapan....@valuelabs.com
<mailto:alapan....@valuelabs.com> > wrote:
Hello Nuwan,
Please find the requested information for backend API directly :
Request Headers --
Source-Identifier: A076FCBB-41F3-4FB5-A9P8-DA2DCF6YR942
Content-Type: application/json
Authorization-Token :
Response Headers --
Cache-Control →no-cache
Content-Length →25078
Content-Type →application/json; charset=utf-8
Date →Tue, 24 Apr 2018 12:11:06 GMT
Expires →-1
Pragma →no-cache
Server →Microsoft-IIS/8.0
X-AspNet-Version →4.0.30319
X-Powered-By →ASP.NET <http://ASP.NET>
Best regards
Alapan Sur
Project Manager - SD
ValueLabs - 'inspired by potential'
Desk: +91-40-6623-9000 | Ext: 32112
Mobile: +91-7799681717
This e-mail may contain information that is proprietary, confidential or
otherwise protected from disclosure and is sent for the intended
recipient(s) only. For email disclaimer details, please visit:
http://www.valuelabs.com/terms-of-use/
From: Nuwan Dias [mailto:nuw...@wso2.com <mailto:nuw...@wso2.com> ]
Sent: Tuesday, April 24, 2018 5:24 PM
To: alapan....@valuelabs.com <mailto:alapan....@valuelabs.com>
Cc: Abimaran Kugathasan <abima...@wso2.com <mailto:abima...@wso2.com> >; dev
<dev@wso2.org <mailto:dev@wso2.org> >
Subject: Re: [Dev] One Specific API not responding
Hi Alan,
Can you send the list of Request headers and Response headers from a
successful request that goes to your back-end directly (without going
through the Gateway)?
Thanks,
Nuwan.
On Tue, Apr 24, 2018 at 5:03 PM, <alapan....@valuelabs.com
<mailto:alapan....@valuelabs.com> > wrote:
Hello Abimaran,
I have increased socket timeout and connection timeout to 3 mints and also
disabled the suspension of API, I am still getting 504 gateway timeout error
and sometime I get the response. Also I have checked the backend API but we
are getting response from backend server within 2 -3 seconds .
I need immediate help to sort this out as we already moved this to
production and customers are facing issue with timeout.
Best regards
Alapan Sur
Project Manager - SD
ValueLabs - 'inspired by potential'
Desk: +91-40-6623-9000 | Ext: 32112
Mobile: +91-7799681717
This e-mail may contain information that is proprietary, confidential or
otherwise protected from disclosure and is sent for the intended
recipient(s) only. For email disclaimer details, please visit:
http://www.valuelabs.com/terms-of-use/
From: Abimaran Kugathasan [mailto:abima...@wso2.com
<mailto:abima...@wso2.com> ]
Sent: Tuesday, April 24, 2018 4:41 PM
To: alapan....@valuelabs.com <mailto:alapan....@valuelabs.com>
Cc: dev <dev@wso2.org <mailto:dev@wso2.org> >
Subject: Re: [Dev] One Specific API not responding
Hi Alapan,
Do you want to increase socket timeout or remove the suspension of API?
On Fri, Apr 20, 2018 at 11:24 AM, <alapan....@valuelabs.com
<mailto:alapan....@valuelabs.com> > wrote:
Hello Team,
Any idea on the below ?
Best regards
Alapan Sur
Project Manager - SD
ValueLabs - 'inspired by potential'
Desk: +91-40-6623-9000 | Ext: 32112
Mobile: +91-7799681717
This e-mail may contain information that is proprietary, confidential or
otherwise protected from disclosure and is sent for the intended
recipient(s) only. For email disclaimer details, please visit:
http://www.valuelabs.com/terms-of-use/
From: alapan....@valuelabs.com <mailto:alapan....@valuelabs.com>
[mailto:alapan....@valuelabs.com <mailto:alapan....@valuelabs.com> ]
Sent: Thursday, April 19, 2018 1:24 PM
To: 'dev@wso2.org <mailto:dev@wso2.org> ' <dev@wso2.org
<mailto:dev@wso2.org> >
Subject: One Specific API not responding
Hello Team,
We are facing an issue file working with WSO2 API Manager 2.1.0.
There is one request for a specific endpoint is continuously failing for
some reason and as a result, this specific API is getting down and we are
not getting an response from WSO2 APIM gateway although we are receiving
the response from backend API before the socket time out error. Please find
the below error we are getting :
TID: [-1] [] [2018-04-19 04:54:24,294] WARN
{org.apache.synapse.transport.passthru.SourceHandler} - Connection time out
after request is read: http-incoming-106261 Socket Timeout : 60000 Remote
Address : /
<https://l.facebook.com/l.php?u=http%3A%2F%2F10.138.120.228%3A36754%2F&h=ATOl7IzK7wN9mVsfkzG0fh6ZtnMR7L9K40MZWk0LYuJEbmNeWspvtYMO4BNt3VE7oDGgr23jwDtmmzAmNzl9jDrCHFvSnNLITEmpFqP6iWx4v_OkfvIqNnIxGpvMv6PB>
10.138.120.228:36754 {org.apache.synapse.transport.passthru.SourceHandler}
<http://l.facebook.com/l.php?u=http%3A%2F%2F10.138.120.228%3A36754%2F&h=ATPsubv-DfKlSaiNx21YeahwUalUYsKADBcz4F-OaA0nNb7IrCxSzP9E4OvrEmhKGyMow8idhB7i0Ij9HBZZRiJUfMgIi7R31JXKm0GM27z_dVvC_69FV9pEg9RG9I6_2afAkKPvJcE&s=1>Is
there any threshold configuration which is configured to handle themultiple
API failure and WSO2 skips suspend the end point ?Can anyone help in this
regard ?Best regardsAlapan SurProject Manager - SDValueLabs - 'inspired by
potential'Desk: +91-40-6623-9000 | Ext: 32112Mobile: +91-7799681717This e-mail
may contain information that is proprietary, confidential orotherwise protected
from disclosure and is sent for the intendedrecipient(s) only. For email
disclaimer details, please
visit:http://www.valuelabs.com/terms-of-use/_______________________________________________Dev
mailing list...@wso2.org
<mailto:Dev@wso2.org>http://wso2.org/cgi-bin/mailman/listinfo/dev--ThanksAbimaran
KugathasanSenior Software Engineer - API TechnologiesEmail : abima...@wso2.com
<mailto:abima
r...@wso2.com>Mobile : +94 773922820
<http://stackoverflow.com/users/515034><http://lk.linkedin.com/in/abimaran>
<http://www.lkabimaran.blogspot.com/><https://github.com/abimarank>
<https://twitter.com/abimaran>_______________________________________________Dev
mailing list...@wso2.org
<mailto:Dev@wso2.org>http://wso2.org/cgi-bin/mailman/listinfo/dev--Nuwan
DiasSoftware Architect - WSO2, Inc. http://wso2.comemail : nuw...@wso2.com
<mailto:nuw...@wso2.com>Phone : +94 777 775 729--Nuwan DiasSoftware Architect -
WSO2, Inc. http://wso2.comemail : nuw...@wso2.com <mailto:nuw...@wso2.com>Phone
: +94 777 775 729--ThanksAbimaran KugathasanSenior Software Engineer - API
TechnologiesEmail : abima...@wso2.com <mailto:abima...@wso2.com>Mobile : +94
773922820
<http://stackoverflow.com/users/515034><http://lk.linkedin.com/in/abimaran>
<http://www.lkabimaran.blogspot.com/><https://github.com/abimarank>
<https://twitter.com/abimaran>_______________________________________________Dev
mailing
list...@wso2.org
<mailto:Dev@wso2.org>http://wso2.org/cgi-bin/mailman/listinfo/dev--Regards,Chamila
AdhikarinayakeSenior Software EngineerWSO2, Inc.Mobile - +94712346437Email -
chami...@wso2.com <mailto:chami...@wso2.com>Blog -
http://helpfromadhi.blogspot.com/_______________________________________________
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev
