Hi,
I'm closing up and opening the connections after execution of insert query
by my bolt.Do I need to increase the max connections db to a level of 500
to 1000? Here in log file of supervisor it tells me the connection made by
bolt is closed and again taking new connection.I think after closing it is
going into ideal and not used for new connections.
2016-05-05T10:44:37.761+0530 STDIO [INFO] COMMAND TO CLOSE CONNECTION IS
DONE
2016-05-05T10:44:37.761+0530 STDIO [INFO] DB connected....
if I'm right what is the solution for those ideal connections?
*Best regards,*
*K.Sai Dilip Reddy.*
On Thu, May 5, 2016 at 1:01 AM, Spico Florin <spicoflo...@gmail.com> wrote:
> hi!
> you have 9 bolts with 50 max db connections. so for each bolt you get a
> conection pool. try to decresae this number for example to 5 and check if
> your performance if fine with your db
> regards,
> florin
>
> On Wednesday, May 4, 2016, Sai Dilip Reddy Kiralam <
> dkira...@aadhya-analytics.com> wrote:
>
>> sorry ! my db is not yet started, so it given me the error ! but when I
>> gave that statement and run the topology then it is using more connections
>> than the specified number.
>>
>>
>>
>> *Best regards,*
>>
>> *K.Sai Dilip Reddy.*
>>
>> On Wed, May 4, 2016 at 11:22 AM, Sai Dilip Reddy Kiralam <
>> dkira...@aadhya-analytics.com> wrote:
>>
>>>
>>> Hi,
>>>
>>> I just added
>>>
>>> a statement
>>> * // config.setMaximumPoolSize(50);*
>>> public synchronized void prepare() {
>>> if(dataSource == null) {
>>> Properties properties = new Properties();
>>> properties.putAll(configMap);
>>> HikariConfig config = new HikariConfig(properties);
>>> * config.setMaximumPoolSize(50);*
>>> this.dataSource = new HikariDataSource(config);
>>> this.dataSource.setAutoCommit(false);
>>> }
>>> }
>>>
>>>
>>> and I'm getting the following error
>>>
>>> java.lang.RuntimeException: Fail-fast during pool initialization at
>>> com.zaxxer.hikari.pool.HikariPool.fillPool(HikariPool.java:475) at
>>> com.zaxxer.hikari.pool.HikariPool.<init>(HikariPool.java:159) at
>>> com.zaxxer.hikari.pool.HikariPool.<init>(HikariPool.java:112) at
>>> com.zaxxer.hikari.HikariDataSource.<init>(HikariDataSource.java:78) at
>>> com.aail.config.MConnectionProvider.prepare(MConnectionProvider.java:27) at
>>> com.aail.storm.bolts.AbstractJdbcBolt.prepare(AbstractJdbcBolt.java:34) at
>>> com.aail.storm.bolts.Inserts.prepare(Inserts.java:290) at
>>> backtype.storm.daemon.executor$fn__6647$fn__6659.invoke(executor.clj:692)
>>> at backtype.storm.util$async_loop$fn__459.invoke(util.clj:461) at
>>> clojure.lang.AFn.run(AFn.java:24) at java.lang.Thread.run(Thread.java:745)
>>> Caused by: org.postgresql.util.PSQLException: Connection attempt timed out.
>>> at org.postgresql.Driver$ConnectThread.getResult(Driver.java:372) at
>>> org.postgresql.Driver.connect(Driver.java:284) at
>>> java.sql.DriverManager.getConnection(DriverManager.java:664) at
>>> java.sql.DriverManager.getConnection(DriverManager.java:247) at
>>> org.postgresql.ds.common.BaseDataSource.getConnection(BaseDataSource.java:99)
>>> at
>>> org.postgresql.ds.common.BaseDataSource.getConnection(BaseDataSource.java:82)
>>> at com.zaxxer.hikari.pool.HikariPool.addConnection(HikariPool.java:398) at
>>> com.zaxxer.hikari.pool.HikariPool.fillPool(HikariPool.java:474) ... 10 more
>>>
>>>
>>>
>>>
>>> *Best regards,*
>>>
>>> *K.Sai Dilip Reddy.*
>>>
>>> On Wed, May 4, 2016 at 9:53 AM, Sai Dilip Reddy Kiralam <
>>> dkira...@aadhya-analytics.com> wrote:
>>>
>>>> Hi,
>>>>
>>>> Thank you Spico Florin.
>>>>
>>>>
>>>>
>>>> *Best regards,*
>>>>
>>>> *K.Sai Dilip Reddy.*
>>>>
>>>> On Wed, May 4, 2016 at 1:18 AM, Spico Florin <spicoflo...@gmail.com>
>>>> wrote:
>>>>
>>>>> hi!
>>>>> please have a look
>>>>> https://github.com/apache/storm/tree/master/external/storm-jdbc where
>>>>> you hve to implemnt thw interface connectionprovider interface. you have
>>>>> to
>>>>> dins a third paty librry that provides conection pooloing for postgresql
>>>>> and use that library to implement the interface connection provider. i
>>>>> hope
>>>>> that other community member will give you some other ideas.
>>>>> i hope that these help.
>>>>> regards,
>>>>> florin
>>>>>
>>>>> On Tuesday, May 3, 2016, Sai Dilip Reddy Kiralam <
>>>>> dkira...@aadhya-analytics.com> wrote:
>>>>>
>>>>>> to implemI'm using storm jdbc for connecting to db.how can I use
>>>>>> pooling on codes on topology.please share information on connection
>>>>>> pooling
>>>>>> used on storm topologies.
>>>>>>
>>>>>>
>>>>>>
>>>>>> *Best regards,*
>>>>>>
>>>>>> *K.Sai Dilip Reddy.*
>>>>>>
>>>>>> On Thu, Apr 28, 2016 at 12:28 PM, Sai Dilip Reddy Kiralam <
>>>>>> dkira...@aadhya-analytics.com> wrote:
>>>>>>
>>>>>>> Hi Spico,
>>>>>>>
>>>>>>> I use 9 bolts with parallelism of 1 and with 1 task for each
>>>>>>> bolt(default) and I'm not using any ConnectionPool for connecting to
>>>>>>> postgres.Just using jdbc classe examples here is source example
>>>>>>> http://storm.apache.org/releases/2.0.0-SNAPSHOT/storm-jdbc.html.
>>>>>>> I will check by using the ConnectionPool.
>>>>>>>
>>>>>>> Thank you.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> *Best regards,*
>>>>>>>
>>>>>>> *K.Sai Dilip Reddy.*
>>>>>>>
>>>>>>> On Thu, Apr 28, 2016 at 12:19 PM, Spico Florin <
>>>>>>> spicoflo...@gmail.com> wrote:
>>>>>>>
>>>>>>>> Hello!
>>>>>>>> How many tasks do you have for inserting the data to your
>>>>>>>> database? Are you using ConnectionPool for connecting to Postgres? If
>>>>>>>> your
>>>>>>>> number of task superseeds the number of max connections provided in
>>>>>>>> connection pool then your have a problem.
>>>>>>>> Please also check the number of max connections that your db
>>>>>>>> accepts.
>>>>>>>> I hope that these help.
>>>>>>>> Regards,
>>>>>>>> Florin
>>>>>>>>
>>>>>>>> On Thu, Apr 28, 2016 at 7:09 AM, Sai Dilip Reddy Kiralam <
>>>>>>>> dkira...@aadhya-analytics.com> wrote:
>>>>>>>>
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>> I have topology that make connection with postgresdb and insert
>>>>>>>>> the fields into tables of a test db.my topology is working fine but
>>>>>>>>> when I
>>>>>>>>> submit the topology it is establishing all the connections of db.but I
>>>>>>>>> don’t know why it is taking all the max connections.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> below attached the pics of pg_stat_activity.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> *Best regards,*
>>>>>>>>>
>>>>>>>>> *K.Sai Dilip Reddy.*
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>
>>>
>>
