Hi,
I made a speed comparison of retrieving data from Apache Ignite using
several methods. All records are in one table, I did not use any WHERE
condition, only a SELECT * FROM TABLE XYZ LIMIT 20000.
Test results are:
Apache Ignite
- Apache Ignite REST API - 0.52 seconds
- JDBC - 4 seconds
- Python pyignite - 40 seconds !!!
pseudocode in Python using pyignite:
client = Client(username="ignite", password="pass", use_ssl=False)
client.connect('localhost', 10800)
cursor=client.sql('SELECT * FROM TABLE_XYZ LIMIT 20000')for row in cursor:
pass
After that I made a speed comparison of retrieving data from PostgreSQL
using JDBC and psycopg2 Python package. SQL select is same, SELECT * FROM
TABLE XYZ LIMIT 20000
PostgreSQL
- JDBC - 3 seconds
- Python psycopg2 using fetchall - 3 seconds
- Python psycopg2 using fetchone - 4 seconds
pseudocode in Python using psycopg2:
import psycopg2
conn = psycopg2.connect(database=DB_NAME,
user=DB_USER,
password=DB_PASS,
host=DB_HOST,
port=DB_PORT)
cur = conn.cursor()
cur.execute("SELECT * FROM TABLE_XYZ LIMIT 20000")
rows = cur.fetchall()for data in rows:
pass
I can conclude that the pyignite implementation has much worse performance
compared to psycopg2 tests. The performance difference on PostgreSQL
between Java JDBC and Python psycopg2 is negligible.
The performance difference on Apache Ignite between Java JDBC and Python
pyignite is very big.
Please if someone can comment on the tests, did I do something wrong or are
these results expected? How can such large differences in execution times
be explained? Do you have any suggestions to get better results using
pyignite?
Thank you
