Re: EPOCH seconds in future!

[dbutkovic]

Hi Ilya,

thnx for replay, I think that Ignite CURRENT_TIMESTAMP() is NOT TIMESTAMP
WITH TIME ZONE.

Below it test with inserting CURRENT_TIMESTAMP in two tables, one with data
type TIMESTAMP and second with TIMESTAMP WITH TIME ZONE.

First of all, 
in Ignite documentation I can't find that CURRENT_TIMESTAMP() returns
TIMESTAMP WITH TIME ZONE.

In H2 documentation we can see that H2 have two data types:
TIMESTAMP and TIMESTAMP WITH TIME ZONE 
http://www.h2database.com/html/datatypes.html#timestamp_with_time_zone_type

In H2 CURRENT_TIMESTAMP returns the current timestamp with time zone.

I think that in Ignite CURRENT_TIMESTAMP is ONLY CURRENT_TIMESTAMP without
TIME ZONE.

https://apacheignite-sql.readme.io/docs/current_timestamp


0: jdbc:ignite:thin://192.168.50.95/> SELECT CURRENT_TIMESTAMP();
++
|  CURRENT_TIMESTAMP()   |
++
| 2020-04-27 13:59:39.814|
++

In first table insert of CURRENT_TIMESTAMP() in data type timestamp is OK.

CREATE TABLE TEST1
(
id  varchar(10),
time1   timestamp,
PRIMARY KEY (id)
) WITH "CACHE_NAME=TEST1, DATA_REGION=PersistDataRegion,
TEMPLATE=REPLICATED, BACKUPS=1";

0: jdbc:ignite:thin://192.168.50.95/> INSERT INTO TEST1 (id, time1) values
('a', CURRENT_TIMESTAMP());
1 row affected (0.051 seconds)


In second table insert of CURRENT_TIMESTAMP() in data type timestamp with
time zone is NOT OK.

CREATE TABLE TEST2
(
id  varchar(10),
time1   timestamp with time zone,
PRIMARY KEY (id)
) WITH "CACHE_NAME


0: jdbc:ignite:thin://192.168.50.95/> INSERT INTO TEST2 (id, time1) values
('a', CURRENT_TIMESTAMP());
Error: class org.apache.ignite.IgniteException: Failed to execute SQL query.
Hexadecimal string with odd number of characters: "2020-04-27 13:59:00.599";
SQL statement:
SELECT
TABLE.ID,
TABLE.TIME1
FROM TABLE(ID VARCHAR(10)=('a',), TIME1 OTHER=(CURRENT_TIMESTAMP(),))
[90003-197] (state=5,code=1)
java.sql.SQLException: class org.apache.ignite.IgniteException: Failed to
execute SQL query. Hexadecimal string with odd number of characters:
"2020-04-27 13:59:00.599"; SQL statement:
SELECT
TABLE.ID,
TABLE.TIME1
FROM TABLE(ID VARCHAR(10)=('a',), TIME1 OTHER=(CURRENT_TIMESTAMP(),))
[90003-197]
at
org.apache.ignite.internal.jdbc.thin.JdbcThinConnection.sendRequest(JdbcThinConnection.java:750)
at
org.apache.ignite.internal.jdbc.thin.JdbcThinStatement.execute0(JdbcThinStatement.java:212)
at
org.apache.ignite.internal.jdbc.thin.JdbcThinStatement.execute(JdbcThinStatement.java:475)
at sqlline.Commands.execute(Commands.java:823)
at sqlline.Commands.sql(Commands.java:733)
at sqlline.SqlLine.dispatch(SqlLine.java:795)
at sqlline.SqlLine.begin(SqlLine.java:668)
at sqlline.SqlLine.start(SqlLine.java:373)
at sqlline.SqlLine.main(SqlLine.java:265)


Please, do you know why I can't insert CURRENT_TIMESTAMP() in data type
timestamp with time zone.


Best regards




--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: EPOCH seconds in future!

[dbutkovic]

Hi Ilya,

thanks a lot for the reply, 
it is surprising  that EPOCH is not always the same regardless of the
timezone. 
I did a test on two Ignite instances, one on a host with timezone 'Europe /
Zagreb' and the other on UTC. 
The EPOCH obtained in bash is the same on both hosts and the EPOCH obtained
in Ignite sql is not the same.

When I select CURRENT_TIMESTAMP() Ignite does not return timezone
information, how do distinct if returned timestamp is local time or UTC?

0: jdbc:ignite:thin://192.168.50.95/> SELECT CURRENT_TIMESTAMP(),
FORMATDATETIME( CURRENT_TIMESTAMP(), '-MM-dd HH:mm:ss z', 'en',
'Europe/Zagreb');
+++
|  CURRENT_TIMESTAMP()   | FORMATDATETIME(CURRENT_TIMESTAMP(),
'-MM-dd HH:mm:ss z', 'en', 'Europe/Zag |
+++
| 2020-04-26 13:49:36.413| 2020-04-26 13:49:36 CEST 
 
|
+++
1 row selected (0.002 seconds)



 

Best regards
Dren



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

EPOCH seconds in future!

[dbutkovic]

Hi,
Ignite function EXTRACT (EPOCH from CURRENT_TIMESTAMP(3)) return seconds in
future!!!


Current date and time on UNIX host, I am in Zagreb/Croatia CEST  GMT+2

[root@incumbossdev01 ~]# date
Fri Apr 17 10:51:10 CEST 2020



Connected to: Apache Ignite (version 2.7.6#20190911-sha1:21f7ca41)
Driver: Apache Ignite Thin JDBC Driver (version
2.7.6#20190911-sha1:21f7ca41)
Autocommit status: true
Transaction isolation: TRANSACTION_REPEATABLE_READ
sqlline version 1.3.0
0: jdbc:ignite:thin://192.168.50.95/> select CURRENT_TIMESTAMP(3);
++
|  CURRENT_TIMESTAMP(3)  |
++
| 2020-04-17 10:51:17.43 |
++
1 row selected (0.032 seconds)


https://apacheignite-sql.readme.io/docs/extract

0: jdbc:ignite:thin://192.168.50.95/> select EXTRACT (EPOCH from
CURRENT_TIMESTAMP(3));
+--+
| EXTRACT(EPOCH FROM CURRENT_TIMESTAMP(3)) |
+--+
| 1587120685.619   |
+--+
1 row selected (0.007 seconds)


Convert EPOCH to Timestamp using https://www.epochconverter.com/

The current Unix epoch time is  1587113657

Assuming that this timestamp is in seconds:
GMT: Friday, 17. April 2020 10:51:25.619
Your time zone: petak, 17. travanj 2020 12:51:25.619 GMT+02:00 DST
Relative: In 2 hours


Convert EPOCH to Timestamp using Postgres function 

postgres=# select to_timestamp(1587120685.619);
to_timestamp

 2020-04-17 12:51:25.619+02
(1 row)



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

EPOCH to timestamp

[dbutkovic]

Hi,
is there a function with which timestamp can be obtained from epoch.

example

select EXTRACT (EPOCH from CURRENT_TIMESTAMP(3));
+--+
| EXTRACT(EPOCH FROM CURRENT_TIMESTAMP(3)) |
+--+
| 1587113983.052   |
+--+

How to convert from epoch seconds to timestamp ?

Best regards
Dren



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Python - Ignite 2.8.0 - java.lang.NullPointerException

[dbutkovic]

Hi Evgenii,
yesterday and in several previous days i was doing some testing on Ignite
2.8.0.
Yesterday i did a fresh install (rm $ IGNITE_HOME/work) on test instance and
now on a freshly started instance I can't reproduce the problem.
For now, we can close the case, if I succeed to reproduce the problem again
I will reopen post.

Best regards

Dren



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

RE: Re: Ignite 2.8.0. Heap mem issue

[dbutkovic]

Hi,

in attachment is code for heap mem issue simulation. Ignite_2.zip
  

# create cache TEST and insert random data into cache TEST
test_insert_in_cache.py

# read data from cache TEST and inset/update data in table TEST_TABLE via
SQL
read_cache_insert_update_table.py


Best regads

Dren




--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Python - Ignite 2.8.0 - java.lang.NullPointerException

[dbutkovic]

I have one small error in py code, please remove last "," from UPDATE
statement in test_insert_update.py

SQL_UPDATE =''' UPDATE TEST SET 
COL1 = ?, 
COL2 = ?,
COL3 = ?,
COL4 = ?,
COL5 = ?,
COL6 = ?,
COL7 = ?,
COL8 = ?,
COL9 = ?,
COL10= ?,
COL11= ?,
COL12= ?
WHERE ID = ? 
''' 

regardless of this sintax error, insert on Ignite 2.8.0 is not ok.




--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Ignite 2.8.0. Heap mem issue

[dbutkovic]

Hi Alex, 

i did another test and collected all the logs, GC logs, Heap mem dump, fiew
screenshots.

All files are in zip file. File is to big for upload, please download from
Jumbo mail link.

https://jumboiskon.tportal.hr/download/eeab9848-2494-4ab7-a2cb-88766db0fafa

Thanks, Dren



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Ignite 2.8.0. Heap mem issue

[dbutkovic]

Hi Evgeny,

I can share pseudo code, not all developed code and python modules.
Syslog messages are inseted into cache via Flume.
Python daemons process messages ( read from CACHE ) and make
insert/update/delete in fiew Ignite tables via SQL.

Number of syslog messages and python code is same on prod and test.

##

# scan CACHE, fetch k/v from CACHE, SQL insert/update/delete ...


while True:

try:
ig_connection = new_ignite_connection(config_params, 
logger)
stream_cache = ig_connection.get_cache("XY_CACHE_NAME") 
# cache is
created by code with get_or_create_cache

CACHE_SIZE = stream_cache.get_size()

if CACHE_SIZE > 0:

# returns a generator, that yields two-tuples 
of key and value
cache_list = stream_cache.scan()

for k, v in cache_list:

# parse k, v and make some 
INSERT/UPDATE in TABLES
# TABLE ALERTS ~ 50 colums
SQL_EVENT_INSERT_QUERY = 
"INSERT INTO ALERTS (%s) VALUES (%s)" %
(alerts_insert_column_names(), alerts_insert_parameters() )


ig_connection.sql(SQL_EVENT_INSERT_QUERY, query_args =
var_EVENT_INSERT_QUERY)

# remove 
stream_cache.remove_key(k)

ig_connection.close()

time.sleep(POLL_INTERVAL)

except (OSError, SocketError) as e:
logger.error('ERROR in client connection: %s , client 
status: %s ' %(e,
ig_connection)) 




##


 




--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: HOW TO CHANGE IGNITE JAVA THIN CLIENT PASSWORD

[dbutkovic]

 I think that username must be in single quote without \

igniteClient.query(new SqlFieldsQuery(" ALTER USER 'ignite' WITH PASSWORD
'password' "));



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: HOW TO CHANGE IGNITE JAVA THIN CLIENT PASSWORD

[dbutkovic]

try with ALTER USER 'ignite' WITH PASSWORD 'test'





--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Ignite 2.8.0. Heap mem issue

[dbutkovic]

Hello,
I recently installed Ignite 2.8.0 on one node for test pourpose and noticed
Heap mem issue that I didn't have on 2.7.6. 
The Ignite configuration of dataStorageConfiguration /
DataRegionConfiguration is identical.
In production I have two nodes (2.7.6) and for test purpose (2.8.0) only one
node.
The application consists of Apache Flume that inserts data into the cache
and several Python scripts that read/write data from/into tables using SQL
queries. 
Does anyone have experience with this behavior on Ignite 2.8.0. ? 
I tested using pyignite 0.3.4 and pygridgain 1.1.0. 
The same issue occurs on GridGain CE 8.7.12. 
On Ignite 2.7.5 with same application there is no issues with Heap mem. 
The screenshots below show that on Ignite 2.7.6 Heap Mem is always within
normal ranges.
One second screenshot Ignite 2.8.0. Heap mem only grows and application
stops when the OOM happens. :-(

JVM_OPTS
-XX:+UseG1GC
-XX:+AlwaysPreTouch
-XX:+ScavengeBeforeFullGC
-XX:+DisableExplicitGC 
-Xms512m -Xmx1g

Metrics for local node (to disable set 'metricsLogFrequency' to 0)
^-- Node [id=1ce47741, name=UMBOSS, uptime=00:46:08.050]
^-- H/N/C [hosts=1, nodes=2, CPUs=8]
^-- CPU [cur=52.07%, avg=12.77%, GC=24.1%]
^-- PageMemory [pages=115217]
^-- Heap [used=1001MB, free=2.15%, comm=1024MB]
^-- Off-heap [used=455MB, free=80.61%, comm=1480MB]
^--   sysMemPlc region [used=0MB, free=99.99%, comm=100MB]
^--   metastoreMemPlc region [used=0MB, free=99.95%, comm=0MB]
^--   PersistDataRegion region [used=440MB, free=56.95%, comm=1024MB]
^--   TxLog region [used=0MB, free=100%, comm=100MB]
^--   DefaultDataRegion region [used=14MB, free=98.59%, comm=256MB]
^-- Ignite persistence [used=479MB]
^--   sysMemPlc region [used=0MB]
^--   metastoreMemPlc region [used=0MB]
^--   PersistDataRegion region [used=479MB]
^--   TxLog region [used=0MB]
^-- Outbound messages queue [size=0]
^-- Public thread pool [active=0, idle=0, qSize=0]
^-- System thread pool [active=1, idle=4, qSize=0]



Best regards

Dren


 



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/