Hello Gordon,
Thank you for the feedback.
> i'm having trouble understanding the issue. the marker (message_id in
> this case) denotes the starting point of query. not sure why you would
> expect message_id4 to show up if message_id3 is newer.
In the case described above message_id3 is not newer than message_id4,
but it follows message_id4 in the lexicographical order (actually
message_id field is uuid string).
In a case of sqlalchemy storage implementation, there is another
internal "id" field that determines an order of events in which they
were added to a database. Please see below events from a real
environment:
+-----+--------------------------------------+-------------------+------+---------------+
| id | message_id | generated | raw
| event_type_id |
+-----+--------------------------------------+-------------------+------+---------------+
| 1 | ecc551cf-99a3-4bfd-b0a2-a1eda8bbaa5c | 1503765208.456666 | {}
| 1 |
| 2 | cf5618e8-b0b8-4277-a947-267349459d74 | 1503765208.480181 | {}
| 2 |
| 3 | 026a86b1-b070-4de5-b988-125b301e5518 | 1503765212.875572 | {}
| 2 |
| 4 | ecf9efd1-83b4-42f7-9689-6f681f0e0001 | 1503765283.018215 | {}
| 3 |
| 5 | 6afabe73-5949-4685-b682-b6b239e5059b | 1503765210.652397 | {}
| 1 |
| 6 | 6f9ad8c1-eeaa-4ff3-8b44-662152967b35 | 1503765210.669995 | {}
| 2 |
| 7 | 12492bef-a8de-464b-a099-41eaf5360ca2 | 1503765212.771545 | {}
| 1 |
| 8 | 935d58e2-c043-49c2-9027-c0e9c54b2c73 | 1503765284.584762 | {}
| 4 |
| 9 | 9b7f0e14-0255-43d6-9398-128d930087b4 | 1503765397.385005 | {}
| 5 |
+-----+--------------------------------------+--------------------------+---------------+
You can see that the event with id 5 was generated earlier than event
3, but it was added to the database after event 4. The same is true
for events 6 and 7.
+----+--------------------------------------+-------------------+------+---------------+
| id | message_id | generated | raw
| event_type_id |
+----+--------------------------------------+-------------------+------+---------------+
| 1 | ecc551cf-99a3-4bfd-b0a2-a1eda8bbaa5c | 1503765208.456666 | {}
| 1 |
| 2 | cf5618e8-b0b8-4277-a947-267349459d74 | 1503765208.480181 | {}
| 2 |
| 5 | 6afabe73-5949-4685-b682-b6b239e5059b | 1503765210.652397 | {}
| 1 |
| 6 | 6f9ad8c1-eeaa-4ff3-8b44-662152967b35 | 1503765210.669995 | {}
| 2 |
| 7 | 12492bef-a8de-464b-a099-41eaf5360ca2 | 1503765212.771545 | {}
| 1 |
| 3 | 026a86b1-b070-4de5-b988-125b301e5518 | 1503765212.875572 | {}
| 2 |
| 4 | ecf9efd1-83b4-42f7-9689-6f681f0e0001 | 1503765283.018215 | {}
| 3 |
| 8 | 935d58e2-c043-49c2-9027-c0e9c54b2c73 | 1503765284.584762 | {}
| 4 |
| 9 | 9b7f0e14-0255-43d6-9398-128d930087b4 | 1503765397.385005 | {}
| 5 |
+----+--------------------------------------+-------------------+------+---------------+
Assume, that I made a request after events 1, 2, 3 and 4 were added to
the database. After some time events 5, 6, 7, 8 and 9 were added to
the database. I made another request to get the rest of events and
received only events 8, 9 because of current sorting used for
pagination.
I look for a way to go through events in order determined by the "id"
field. The "id" field is not exposed via api because it is internal
part of storage implementation. We can't use it in api calls.
So, I proposed cursor api as an independent from storage
implementation way to get events in the order they appear in a
database.
> is cursor code part of oslo.db? the one issue i see is we'll need to
> change api results which means we need to do some versioning changes as
> well (not a blocker, just another discussion and more work)
Actually, cursor is not more than the pagenate_query from oslo.db. It
just uses the "id" field as a sort key.
Can I share patch with the prototype, maybe it will help in the discussion?
Thanks,
Mike
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
