Re: [Interest] Qt Sql BLOB data converting
On Monday, 28 November 2022 14:29:30 PST Scott Bloom wrote: > While this might be the expectation from the Qt side of the world đ On > windows, I donât know of anyone who does this. Not so uncommon on linux, > but "all" 3rd party? I wouldnât say that happens often That's not even the expectation from all Qt developers. It is however *my* recommendation. It's required if you need a specific build option that wasn't enabled by Qt. Which is also the case when you must apply a security patch because a vulnerability was found in said library. Since you usually must do that before the next Qt minor release becomes available, you should probably know which third parties you have and have a working build environment for them, instead of having to scramble when a vulnerability is found. -- Thiago Macieira - thiago.macieira (AT) intel.com Cloud Software Architect - Intel DCAI Cloud Engineering ___ Interest mailing list Interest@qt-project.org https://lists.qt-project.org/listinfo/interest
Re: [Interest] Qt Sql BLOB data converting
-Original Message- From: Interest On Behalf Of Thiago Macieira Sent: Monday, November 28, 2022 1:41 PM To: interest@qt-project.org Subject: Re: [Interest] Qt Sql BLOB data converting On Monday, 28 November 2022 12:39:05 PST Scott Bloom wrote: > Strict tables were only added in Sqlite 3.37, I don't know what > version of > sqlite3 is used in Qt6, my version of Qt5.15.6 doesn't support it. Hence the rule of thumb: ALWAYS unbundle the third party content and build the necessary libraries yourself. The third-party content that gets bundled into the Qt libraries should never be deployed to customers. -- While this might be the expectation from the Qt side of the world đ On windows, I donât know of anyone who does this. Not so uncommon on linux, but "all" 3rd party? I wouldnât say that happens often Scott ___ Interest mailing list Interest@qt-project.org https://lists.qt-project.org/listinfo/interest
Re: [Interest] Qt Sql BLOB data converting
On Monday, 28 November 2022 12:39:05 PST Scott Bloom wrote: > Strict tables were only added in Sqlite 3.37, I don't know what version of > sqlite3 is used in Qt6, my version of Qt5.15.6 doesn't support it. Hence the rule of thumb: ALWAYS unbundle the third party content and build the necessary libraries yourself. The third-party content that gets bundled into the Qt libraries should never be deployed to customers. -- Thiago Macieira - thiago.macieira (AT) intel.com Cloud Software Architect - Intel DCAI Cloud Engineering ___ Interest mailing list Interest@qt-project.org https://lists.qt-project.org/listinfo/interest
Re: [Interest] Qt Sql BLOB data converting
-Original Message- From: Interest On Behalf Of Thiago Macieira Sent: Monday, November 28, 2022 12:21 PM To: interest@qt-project.org Subject: Re: [Interest] Qt Sql BLOB data converting On Monday, 28 November 2022 12:07:22 PST Scott Bloom wrote: > but if they did a create XXX as string, insert XXX=123 (integer) and > Sqlite can return it as a string or an integer, the QVariant should be > created from an string not an integer. The same goes if they did a > create XXX as integer, insert XXX='123'. QVariant should be created > as from an integer type, and they can do a toString on the variant if they > want. > > My point, dynamic types should follow C++ style "dynamic" typing not > "everything is a string until proven otherwise" typing of other languages. I don't agree. Sqlite seems to have made a choice that the type is stored per row, but gives you the ability to choose otherwise either at table creation time or at selection time. Changing the behaviour right now would remove that possibility. I don't know if it is surprising or not for unwary people. But I can tell you that no action will happen because QtSql is right now under-maintained and there are no volunteers to understand all the implications of such a choice. I understand the "understaffed" issue, however, to me its problematic that the record method returns a different type than the row. It s been this way for sometime. In fact, I brought this up with the Sql team back in the Qt 4 days. Strict tables were only added in Sqlite 3.37, I don't know what version of sqlite3 is used in Qt6, my version of Qt5.15.6 doesn't support it. This issue, of a blob interpreted as a string has existed for ages. I just think it's a mistake to determine the type of a column in two different ways. Scott ___ Interest mailing list Interest@qt-project.org https://lists.qt-project.org/listinfo/interest
Re: [Interest] Qt Sql BLOB data converting
On Monday, 28 November 2022 12:07:22 PST Scott Bloom wrote: > but if they did a create XXX as string, insert XXX=123 (integer) and Sqlite > can return it as a string or an integer, the QVariant should be created > from an string not an integer. The same goes if they did a create XXX as > integer, insert XXX='123'. QVariant should be created as from an integer > type, and they can do a toString on the variant if they want. > > My point, dynamic types should follow C++ style "dynamic" typing not > "everything is a string until proven otherwise" typing of other languages. I don't agree. Sqlite seems to have made a choice that the type is stored per row, but gives you the ability to choose otherwise either at table creation time or at selection time. Changing the behaviour right now would remove that possibility. I don't know if it is surprising or not for unwary people. But I can tell you that no action will happen because QtSql is right now under-maintained and there are no volunteers to understand all the implications of such a choice. -- Thiago Macieira - thiago.macieira (AT) intel.com Cloud Software Architect - Intel DCAI Cloud Engineering ___ Interest mailing list Interest@qt-project.org https://lists.qt-project.org/listinfo/interest
Re: [Interest] Qt Sql BLOB data converting
-Original Message- From: Interest On Behalf Of Christian Ehrlicher Sent: Monday, November 28, 2022 11:54 AM To: interest@qt-project.org Subject: Re: [Interest] Qt Sql BLOB data converting Am 28.11.2022 um 20:31 schrieb Scott Bloom: > -Original Message- > From: Interest On Behalf Of Thiago > Macieira > Sent: Monday, November 28, 2022 11:21 AM > To: interest@qt-project.org > Subject: Re: [Interest] Qt Sql BLOB data converting > > On Monday, 28 November 2022 10:59:47 PST Christian Ehrlicher wrote: >> It's because of the dynamic typing feature of sqlite ( >> https://www.sqlite.org/flextypegood.html ) - even a column is created >> as one type it can contain another one. Looks like the data was >> inserted with the wrong type information since your blob data is for >> sure no readable string. > Thanks for the explanation, Christian. I can confirm that the type comes out > correctly if it is inserted into the table AS blob, not as plaintext. > Therefore, the problem is on the insertion side, not in the SELECT or QtSql. > > I've closed the task. > > -- > I disagree with closing it. > > When the cache is being created, it should use the same method for > determining the type of a column as the record function does. > > First check the name of the column type, then use the enumerated type if the > string name is empty. This will not work when you fetch two rows with different dynamic types for a column - there is only one record per result set. But at least we should be consistent by directly using the already calculated values from the record instead calling sqlite3_column_type() again. Christian --- If the user/developer creates a table, where the type of a column "Actually" changes per row, they should use a string type for the row, and do the conversion outside of SQL, OR as you posted earlier cast the column on an individual row select command. But if the column is defined as a BLOB the variant should be created from a QByteArray every time, if it's a text then QString, INT integer etc etc. but if they did a create XXX as string, insert XXX=123 (integer) and Sqlite can return it as a string or an integer, the QVariant should be created from an string not an integer. The same goes if they did a create XXX as integer, insert XXX='123'. QVariant should be created as from an integer type, and they can do a toString on the variant if they want. My point, dynamic types should follow C++ style "dynamic" typing not "everything is a string until proven otherwise" typing of other languages. Scott ___ Interest mailing list Interest@qt-project.org https://lists.qt-project.org/listinfo/interest
Re: [Interest] Qt Sql BLOB data converting
Am 28.11.2022 um 20:31 schrieb Scott Bloom: -Original Message- From: Interest On Behalf Of Thiago Macieira Sent: Monday, November 28, 2022 11:21 AM To: interest@qt-project.org Subject: Re: [Interest] Qt Sql BLOB data converting On Monday, 28 November 2022 10:59:47 PST Christian Ehrlicher wrote: It's because of the dynamic typing feature of sqlite ( https://www.sqlite.org/flextypegood.html ) - even a column is created as one type it can contain another one. Looks like the data was inserted with the wrong type information since your blob data is for sure no readable string. Thanks for the explanation, Christian. I can confirm that the type comes out correctly if it is inserted into the table AS blob, not as plaintext. Therefore, the problem is on the insertion side, not in the SELECT or QtSql. I've closed the task. -- I disagree with closing it. When the cache is being created, it should use the same method for determining the type of a column as the record function does. First check the name of the column type, then use the enumerated type if the string name is empty. This will not work when you fetch two rows with different dynamic types for a column - there is only one record per result set. But at least we should be consistent by directly using the already calculated values from the record instead calling sqlite3_column_type() again. Christian ___ Interest mailing list Interest@qt-project.org https://lists.qt-project.org/listinfo/interest
Re: [Interest] Qt Sql BLOB data converting
-Original Message- From: Interest On Behalf Of Thiago Macieira Sent: Monday, November 28, 2022 11:21 AM To: interest@qt-project.org Subject: Re: [Interest] Qt Sql BLOB data converting On Monday, 28 November 2022 10:59:47 PST Christian Ehrlicher wrote: > It's because of the dynamic typing feature of sqlite ( > https://www.sqlite.org/flextypegood.html ) - even a column is created > as one type it can contain another one. Looks like the data was > inserted with the wrong type information since your blob data is for > sure no readable string. Thanks for the explanation, Christian. I can confirm that the type comes out correctly if it is inserted into the table AS blob, not as plaintext. Therefore, the problem is on the insertion side, not in the SELECT or QtSql. I've closed the task. -- I disagree with closing it. When the cache is being created, it should use the same method for determining the type of a column as the record function does. First check the name of the column type, then use the enumerated type if the string name is empty. Scott ___ Interest mailing list Interest@qt-project.org https://lists.qt-project.org/listinfo/interest
Re: [Interest] Qt Sql BLOB data converting
Just a POINT of note. Since SQL Lite has moved to âflexible typingâ (its been
a while) they have also added a strict datatype table
Which I have been using myself.
For this case, rather than
CREATE TABLE IF NOT EXISTS "test" (
"value" BLOB NOT NULL
);
You would use
CREATE TABLE IF NOT EXISTS "test" (
"value" BLOB NOT NULL
) STRICT;
Then all types are returned as defined, and inserts may fail (IMO correctly)
whereas before they would succeed.
I have found creating strict tables correct this an other issues (such as a
string that has the value 1234 being automatically converted to an integer,
and then the value.toString() returning â1234â)
Scott
From: Yauheni Pervenenka
Sent: Monday, November 28, 2022 11:16 AM
To: Scott Bloom
Cc: Christian Ehrlicher ; interest@qt-project.org
Subject: Re: [Interest] Qt Sql BLOB data converting
Scott, thanks you for your time
On Mon, Nov 28, 2022 at 10:11 PM Scott Bloom
mailto:sc...@towel42.com>> wrote:
@Christian Ehrlicher you beat me to the punch.
If you dump out the value (not the size) you will see the QVariant is a of type
QString not QByteArray
const auto value = query.value(0);
qDebug() << "value = " << value;
auto tmp = QVariant( value.toByteArray() );
qDebug() << "tmp = " << tmp;
value = QVariant(QString, "v10I-F½L\u009C{â?x\u009B)δ\u001EG#?fĂłu??H")
tmp = QVariant(QByteArray,
"v10I-F\xC2\xABL\xC2\x9C{\xC2\xB0\xEF\xBF\xBDx\xC2\x9B)\xC3\xAB\x1EG#\xDE\xA6\xEF\xBF\xBD\xEF\xBF\xBD\xEF\xBF\xBD\xEF\xBF\xBD""f\xC2\xA2u\xEF\xBF\xBD\xEF\xBF\xBD\xC4\xA6")
This is the root cause of the problem.
Ive hit this before, the query.value function should return a QByteArray type
of QVariant, not a QString type when the column type is of type blob.
auto record = db.record( "test" );
qDebug() << "record =" << record;
returns the correct record type
record = QSqlRecord(1)
0: QSqlField("value", QByteArray, tableName: "test", required: yes, generated:
yes, autoValue: false, readOnly: false) ""
Whats interesting, is in my qsql_sqlite.cpp (the driver for the sql database,
sqlite3_column_decltype16 returns "BLOB" but sqlite3_column_type returns
SQLITE_TEXT
in the init columns function, to determine which Qt type to use, it first gets
the string name of the column, if empty then use sqlite3_column_type otherwise
it uses qGetColumnType
So the Record is correct, but the return type is of QString not QByteArray
Scott
-Original Message-
From: Interest
mailto:interest-boun...@qt-project.org>> On
Behalf Of Christian Ehrlicher
Sent: Monday, November 28, 2022 10:41 AM
To: Yauheni Pervenenka
mailto:yauheni.pervene...@viber.com>>;
interest@qt-project.org
Subject: Re: [Interest] Qt Sql BLOB data converting
Am 28.11.2022 um 19:16 schrieb Yauheni Pervenenka via Interest:
> Thiago, hello, thanks for your reply, sorry have no instructions for
> create and insert, but have data base with one value which can't be
> parsed correct(db is cut to one value for provide example)
>
> https://bugreports.qt.io/browse/QTBUG-108992
>
Your database looks strange to me - even though your column type according to
the schema is BLOB:
sqlite> .schema
CREATE TABLE IF NOT EXISTS "test" (
"value" BLOB NOT NULL
);
the return value of sqlite3_column_type() is SQLITE_TEXT (3).
Therefore the return value is treated as string and the data is screwed up. If
I change the return value to SQLITE_BLOB (4) in the debugger, all is correct:
qDebug() << value.toByteArray().toHex();
-->
"763130492d46ab4c9c7bb0e9789b29c3ab1e4723dea6c8e5d2f9b266a275e1fbe4a6"
Christian
___
Interest mailing list
Interest@qt-project.org
https://lists.qt-project.org/listinfo/interest
___
Interest mailing list
Interest@qt-project.org
https://lists.qt-project.org/listinfo/interest
Re: [Interest] Qt Sql BLOB data converting
On Monday, 28 November 2022 10:59:47 PST Christian Ehrlicher wrote: > It's because of the dynamic typing feature of sqlite ( > https://www.sqlite.org/flextypegood.html ) - even a column is created as > one type it can contain another one. Looks like the data was inserted > with the wrong type information since your blob data is for sure no > readable string. Thanks for the explanation, Christian. I can confirm that the type comes out correctly if it is inserted into the table AS blob, not as plaintext. Therefore, the problem is on the insertion side, not in the SELECT or QtSql. I've closed the task. -- Thiago Macieira - thiago.macieira (AT) intel.com Cloud Software Architect - Intel DCAI Cloud Engineering ___ Interest mailing list Interest@qt-project.org https://lists.qt-project.org/listinfo/interest
Re: [Interest] Qt Sql BLOB data converting
Scott, thanks you for your time
On Mon, Nov 28, 2022 at 10:11 PM Scott Bloom wrote:
> @Christian Ehrlicher you beat me to the punch.
>
> If you dump out the value (not the size) you will see the QVariant is a of
> type QString not QByteArray
>
> const auto value = query.value(0);
> qDebug() << "value = " << value;
> auto tmp = QVariant( value.toByteArray() );
> qDebug() << "tmp = " << tmp;
>
> value = QVariant(QString, "v10I-F½L\u009C{â?x\u009B)δ\u001EG#?fĂłu??H")
> tmp = QVariant(QByteArray,
> "v10I-F\xC2\xABL\xC2\x9C{\xC2\xB0\xEF\xBF\xBDx\xC2\x9B)\xC3\xAB\x1EG#\xDE\xA6\xEF\xBF\xBD\xEF\xBF\xBD\xEF\xBF\xBD\xEF\xBF\xBD""f\xC2\xA2u\xEF\xBF\xBD\xEF\xBF\xBD\xC4\xA6")
>
> This is the root cause of the problem.
>
> Ive hit this before, the query.value function should return a QByteArray
> type of QVariant, not a QString type when the column type is of type blob.
>
> auto record = db.record( "test" );
> qDebug() << "record =" << record;
>
> returns the correct record type
>
> record = QSqlRecord(1)
> 0: QSqlField("value", QByteArray, tableName: "test", required: yes,
> generated: yes, autoValue: false, readOnly: false) ""
>
> Whats interesting, is in my qsql_sqlite.cpp (the driver for the sql
> database,
>
> sqlite3_column_decltype16 returns "BLOB" but sqlite3_column_type returns
> SQLITE_TEXT
>
> in the init columns function, to determine which Qt type to use, it first
> gets the string name of the column, if empty then use sqlite3_column_type
> otherwise it uses qGetColumnType
>
> So the Record is correct, but the return type is of QString not QByteArray
>
> Scott
>
>
>
>
> -Original Message-
> From: Interest On Behalf Of Christian
> Ehrlicher
> Sent: Monday, November 28, 2022 10:41 AM
> To: Yauheni Pervenenka ;
> interest@qt-project.org
> Subject: Re: [Interest] Qt Sql BLOB data converting
>
>
> Am 28.11.2022 um 19:16 schrieb Yauheni Pervenenka via Interest:
> > Thiago, hello, thanks for your reply, sorry have no instructions for
> > create and insert, but have data base with one value which can't be
> > parsed correct(db is cut to one value for provide example)
> >
> > https://bugreports.qt.io/browse/QTBUG-108992
> >
> Your database looks strange to me - even though your column type according
> to the schema is BLOB:
>
> sqlite> .schema
> CREATE TABLE IF NOT EXISTS "test" (
> "value" BLOB NOT NULL
> );
>
> the return value of sqlite3_column_type() is SQLITE_TEXT (3).
>
> Therefore the return value is treated as string and the data is screwed
> up. If I change the return value to SQLITE_BLOB (4) in the debugger, all is
> correct:
>
> qDebug() << value.toByteArray().toHex();
>
> -->
>
> "763130492d46ab4c9c7bb0e9789b29c3ab1e4723dea6c8e5d2f9b266a275e1fbe4a6"
>
>
>
> Christian
>
> ___
> Interest mailing list
> Interest@qt-project.org
> https://lists.qt-project.org/listinfo/interest
>
___
Interest mailing list
Interest@qt-project.org
https://lists.qt-project.org/listinfo/interest
Re: [Interest] Qt Sql BLOB data converting
@Christian Ehrlicher you beat me to the punch.
If you dump out the value (not the size) you will see the QVariant is a of type
QString not QByteArray
const auto value = query.value(0);
qDebug() << "value = " << value;
auto tmp = QVariant( value.toByteArray() );
qDebug() << "tmp = " << tmp;
value = QVariant(QString, "v10I-F½L\u009C{â?x\u009B)δ\u001EG#?fĂłu??H")
tmp = QVariant(QByteArray,
"v10I-F\xC2\xABL\xC2\x9C{\xC2\xB0\xEF\xBF\xBDx\xC2\x9B)\xC3\xAB\x1EG#\xDE\xA6\xEF\xBF\xBD\xEF\xBF\xBD\xEF\xBF\xBD\xEF\xBF\xBD""f\xC2\xA2u\xEF\xBF\xBD\xEF\xBF\xBD\xC4\xA6")
This is the root cause of the problem.
Ive hit this before, the query.value function should return a QByteArray type
of QVariant, not a QString type when the column type is of type blob.
auto record = db.record( "test" );
qDebug() << "record =" << record;
returns the correct record type
record = QSqlRecord(1)
0: QSqlField("value", QByteArray, tableName: "test", required: yes, generated:
yes, autoValue: false, readOnly: false) ""
Whats interesting, is in my qsql_sqlite.cpp (the driver for the sql database,
sqlite3_column_decltype16 returns "BLOB" but sqlite3_column_type returns
SQLITE_TEXT
in the init columns function, to determine which Qt type to use, it first gets
the string name of the column, if empty then use sqlite3_column_type otherwise
it uses qGetColumnType
So the Record is correct, but the return type is of QString not QByteArray
Scott
-Original Message-
From: Interest On Behalf Of Christian
Ehrlicher
Sent: Monday, November 28, 2022 10:41 AM
To: Yauheni Pervenenka ; interest@qt-project.org
Subject: Re: [Interest] Qt Sql BLOB data converting
Am 28.11.2022 um 19:16 schrieb Yauheni Pervenenka via Interest:
> Thiago, hello, thanks for your reply, sorry have no instructions for
> create and insert, but have data base with one value which can't be
> parsed correct(db is cut to one value for provide example)
>
> https://bugreports.qt.io/browse/QTBUG-108992
>
Your database looks strange to me - even though your column type according to
the schema is BLOB:
sqlite> .schema
CREATE TABLE IF NOT EXISTS "test" (
   "value" BLOB NOT NULL
);
the return value of sqlite3_column_type() is SQLITE_TEXT (3).
Therefore the return value is treated as string and the data is screwed up. If
I change the return value to SQLITE_BLOB (4) in the debugger, all is correct:
qDebug() << value.toByteArray().toHex();
-->
"763130492d46ab4c9c7bb0e9789b29c3ab1e4723dea6c8e5d2f9b266a275e1fbe4a6"
Christian
___
Interest mailing list
Interest@qt-project.org
https://lists.qt-project.org/listinfo/interest
___
Interest mailing list
Interest@qt-project.org
https://lists.qt-project.org/listinfo/interest
Re: [Interest] Qt Sql BLOB data converting
Christian, thanks a lot it is work now, fine On Mon, Nov 28, 2022 at 10:01 PM Christian Ehrlicher wrote: > > Am 28.11.2022 um 19:46 schrieb Yauheni Pervenenka: > > Christian, thanks for your effort, db was simplified for test, is it > > possible to fix it on the client side? Any ideas, why is column > > considered as SQLITE_TEXT if it is BLOB? > > > It's because of the dynamic typing feature of sqlite ( > https://www.sqlite.org/flextypegood.html ) - even a column is created as > one type it can contain another one. Looks like the data was inserted > with the wrong type information since your blob data is for sure no > readable string. > > A cast seems to work: "SELECT cast(value AS BLOB) from test" > > > Christian > > ___ > Interest mailing list > Interest@qt-project.org > https://lists.qt-project.org/listinfo/interest > ___ Interest mailing list Interest@qt-project.org https://lists.qt-project.org/listinfo/interest
Re: [Interest] Qt Sql BLOB data converting
Am 28.11.2022 um 19:46 schrieb Yauheni Pervenenka: Christian, thanks for your effort, db was simplified for test, is it possible to fix it on the client side? Any ideas, why is column considered as SQLITE_TEXT if it is BLOB? It's because of the dynamic typing feature of sqlite ( https://www.sqlite.org/flextypegood.html ) - even a column is created as one type it can contain another one. Looks like the data was inserted with the wrong type information since your blob data is for sure no readable string. A cast seems to work: "SELECT cast(value AS BLOB) from test" Christian ___ Interest mailing list Interest@qt-project.org https://lists.qt-project.org/listinfo/interest
Re: [Interest] Qt Sql BLOB data converting
I used sqlite3 python module, and it is parsed fine On Mon, Nov 28, 2022 at 9:46 PM Yauheni Pervenenka < yauheni.pervene...@viber.com> wrote: > Christian, thanks for your effort, db was simplified for test, is it > possible to fix it on the client side? Any ideas, why is column considered > as SQLITE_TEXT if it is BLOB? > > On Mon, Nov 28, 2022 at 9:40 PM Christian Ehrlicher > wrote: > >> >> Am 28.11.2022 um 19:16 schrieb Yauheni Pervenenka via Interest: >> > Thiago, hello, thanks for your reply, sorry have no instructions for >> > create and insert, but have data base with one value which can't be >> > parsed correct(db is cut to one value for provide example) >> > >> > https://bugreports.qt.io/browse/QTBUG-108992 >> > >> Your database looks strange to me - even though your column type >> according to the schema is BLOB: >> >> sqlite> .schema >> CREATE TABLE IF NOT EXISTS "test" ( >> "value" BLOB NOT NULL >> ); >> >> the return value of sqlite3_column_type() is SQLITE_TEXT (3). >> >> Therefore the return value is treated as string and the data is screwed >> up. If I change the return value to SQLITE_BLOB (4) in the debugger, all >> is correct: >> >> qDebug() << value.toByteArray().toHex(); >> >> --> >> >> "763130492d46ab4c9c7bb0e9789b29c3ab1e4723dea6c8e5d2f9b266a275e1fbe4a6" >> >> >> >> Christian >> >> ___ Interest mailing list Interest@qt-project.org https://lists.qt-project.org/listinfo/interest
Re: [Interest] Qt Sql BLOB data converting
Christian, thanks for your effort, db was simplified for test, is it possible to fix it on the client side? Any ideas, why is column considered as SQLITE_TEXT if it is BLOB? On Mon, Nov 28, 2022 at 9:40 PM Christian Ehrlicher wrote: > > Am 28.11.2022 um 19:16 schrieb Yauheni Pervenenka via Interest: > > Thiago, hello, thanks for your reply, sorry have no instructions for > > create and insert, but have data base with one value which can't be > > parsed correct(db is cut to one value for provide example) > > > > https://bugreports.qt.io/browse/QTBUG-108992 > > > Your database looks strange to me - even though your column type > according to the schema is BLOB: > > sqlite> .schema > CREATE TABLE IF NOT EXISTS "test" ( > "value" BLOB NOT NULL > ); > > the return value of sqlite3_column_type() is SQLITE_TEXT (3). > > Therefore the return value is treated as string and the data is screwed > up. If I change the return value to SQLITE_BLOB (4) in the debugger, all > is correct: > > qDebug() << value.toByteArray().toHex(); > > --> > > "763130492d46ab4c9c7bb0e9789b29c3ab1e4723dea6c8e5d2f9b266a275e1fbe4a6" > > > > Christian > > ___ Interest mailing list Interest@qt-project.org https://lists.qt-project.org/listinfo/interest
Re: [Interest] Qt Sql BLOB data converting
Am 28.11.2022 um 19:40 schrieb Christian Ehrlicher: Am 28.11.2022 um 19:16 schrieb Yauheni Pervenenka via Interest: Thiago, hello, thanks for your reply, sorry have no instructions for create and insert, but have data base with one value which can't be parsed correct(db is cut to one value for provide example) https://bugreports.qt.io/browse/QTBUG-108992 Your database looks strange to me - even though your column type according to the schema is BLOB: sqlite> .schema CREATE TABLE IF NOT EXISTS "test" (    "value" BLOB NOT NULL ); the return value of sqlite3_column_type() is SQLITE_TEXT (3). Therefore the return value is treated as string and the data is screwed up. If I change the return value to SQLITE_BLOB (4) in the debugger, all is correct: qDebug() << value.toByteArray().toHex(); --> "763130492d46ab4c9c7bb0e9789b29c3ab1e4723dea6c8e5d2f9b266a275e1fbe4a6" Even the sqlite command line utility is confused: sqlite> select sql from sqlite_schema where tbl_name = "test"; CREATE TABLE "test" (    "value" BLOB NOT NULL ) sqlite> select typeof(value) from test; text Christian ___ Interest mailing list Interest@qt-project.org https://lists.qt-project.org/listinfo/interest
Re: [Interest] Qt Sql BLOB data converting
Am 28.11.2022 um 19:16 schrieb Yauheni Pervenenka via Interest: Thiago, hello, thanks for your reply, sorry have no instructions for create and insert, but have data base with one value which can't be parsed correct(db is cut to one value for provide example) https://bugreports.qt.io/browse/QTBUG-108992 Your database looks strange to me - even though your column type according to the schema is BLOB: sqlite> .schema CREATE TABLE IF NOT EXISTS "test" (    "value" BLOB NOT NULL ); the return value of sqlite3_column_type() is SQLITE_TEXT (3). Therefore the return value is treated as string and the data is screwed up. If I change the return value to SQLITE_BLOB (4) in the debugger, all is correct: qDebug() << value.toByteArray().toHex(); --> "763130492d46ab4c9c7bb0e9789b29c3ab1e4723dea6c8e5d2f9b266a275e1fbe4a6" Christian ___ Interest mailing list Interest@qt-project.org https://lists.qt-project.org/listinfo/interest
Re: [Interest] Qt Sql BLOB data converting
Thiago, hello, thanks for your reply, sorry have no instructions for create
and insert, but have data base with one value which can't be parsed
correct(db is cut to one value for provide example)
https://bugreports.qt.io/browse/QTBUG-108992
On Mon, Nov 28, 2022 at 9:07 PM Thiago Macieira
wrote:
> On Monday, 28 November 2022 01:00:01 PST Yauheni Pervenenka via Interest
> wrote:
> > QSqlQuery query("SELECT value from table");
> > while (query.next()) {
> > const auto value = query.value(0);
> > qDebug() << value.toByteArray().size(); // size 52
> > qDebug() << value.toString().size(); // size 30
> > qDebug() << value.toByteArray().toHex(); // size 104
> > }
>
> If your data is binary, don't use toString(). That's definitely wrong.
>
> Can you show the example from the CREATE TABLE command, including the
> INSERT
> one?
>
> --
> Thiago Macieira - thiago.macieira (AT) intel.com
> Cloud Software Architect - Intel DCAI Cloud Engineering
>
>
>
> ___
> Interest mailing list
> Interest@qt-project.org
> https://lists.qt-project.org/listinfo/interest
>
___
Interest mailing list
Interest@qt-project.org
https://lists.qt-project.org/listinfo/interest
Re: [Interest] Qt Sql BLOB data converting
On Monday, 28 November 2022 01:00:01 PST Yauheni Pervenenka via Interest
wrote:
> QSqlQuery query("SELECT value from table");
> while (query.next()) {
> const auto value = query.value(0);
> qDebug() << value.toByteArray().size(); // size 52
> qDebug() << value.toString().size(); // size 30
> qDebug() << value.toByteArray().toHex(); // size 104
> }
If your data is binary, don't use toString(). That's definitely wrong.
Can you show the example from the CREATE TABLE command, including the INSERT
one?
--
Thiago Macieira - thiago.macieira (AT) intel.com
Cloud Software Architect - Intel DCAI Cloud Engineering
___
Interest mailing list
Interest@qt-project.org
https://lists.qt-project.org/listinfo/interest
Re: [Interest] Qt Sql BLOB data converting
Hello, Henry, thanks for the reply, seems like no, there is simple code
QSqlQuery query("SELECT value from table");
while (query.next()) {
const auto value = query.value(0);
qDebug() << value.toByteArray().size(); // size 52
qDebug() << value.toString().size(); // size 30
qDebug() << value.toByteArray().toHex(); // size 104
}
On Mon, Nov 28, 2022 at 11:49 AM Henry Skoglund wrote:
> On 2022-11-28 09:32, Yauheni Pervenenka via Interest wrote:
>
> Hi, Tony, thanks for your reply,
> OS: win 11
> DB: SQLite
>
> Unfortunately, QVariant::toByteArray returns size 52(hex 104),
> QVariant::toString returns size 30
>
> There is data returned by QVariant::toByteArray::toHex
> Qt data in hex(size 104):
>
> 763130492d46c2ab4cc29c7bc2b0efbfbd78c29b29c3ab1e4723dea6efbfbdefbfbdefbfbdefbfbd66c2a275efbfbdefbfbdc4a6
>
> Original value in hex
> size(68): 763130492d46ab4c9c7bb0e9789b29c3ab1e4723dea6c8e5d2f9b266a275e1fbe4a6
>
> there are some data that is parsed correct but time to time it is broken
>
>
>
> On Mon, Nov 28, 2022 at 11:17 AM Tony Rietwyk
> wrote:
>
>> Hi Yauheni,
>>
>> You don't say which type of database you are using? There may be issues
>> with the database client library altering the blob to UTF8. Though I
>> would expect the python code using the same client to have the same
>> problem.
>>
>> Assuming that QVariant has the byte array unaltered, then
>> QVariant.toByteArray().length() should give 34. Using qstrlen, suggests
>> that you are using QVariant.toString instead. To print the hex you
>> should just use QVariant.toByteArray().toHex().
>>
>> Hope that helps,
>>
>> Tony
>>
>>
>> On 28/11/2022 6:46 pm, Yauheni Pervenenka via Interest wrote:
>> > Hello, community
>> > I have faced with issue that qvariant using qstrlen for getting data
>> > size, but sometimes it returns wrong size especially for BLOB data(if
>> > there are some special symbols), are there any way to get data correct?
>> >
>> > Note:
>> > Database value 34, python returns correct data size, but
>> > qvariant::toByteArray returns 52
>> >
>> > Python data and Database data in hex(size
>> >
>> 68): 763130492d46ab4c9c7bb0e9789b29c3ab1e4723dea6c8e5d2f9b266a275e1fbe4a6
>> >
>> > Qt data in hex(size 104):
>> >
>> 763130492d46c2ab4cc29c7bc2b0efbfbd78c29b29c3ab1e4723dea6efbfbdefbfbdefbfbdefbfbd66c2a275efbfbdefbfbdc4a6
>> >
>>
>
> Hi, most of the data is preserved in Qt but some other data is injected
> midstream.
> Just a guess, but maybe the Qt version has a multithreading issue, like
> two threads are settings values in the same array at the same time.
>
>
>
> ___
> Interest mailing list
> Interest@qt-project.org
> https://lists.qt-project.org/listinfo/interest
>
___
Interest mailing list
Interest@qt-project.org
https://lists.qt-project.org/listinfo/interest
Re: [Interest] Qt Sql BLOB data converting
On 2022-11-28 09:32, Yauheni Pervenenka via Interest wrote: Hi, Tony, thanks for your reply, OS: win 11 DB: SQLite Unfortunately, QVariant::toByteArray returns size 52(hex 104), QVariant::toString returns size 30 There is data returned by QVariant::toByteArray::toHex Qt data in hex(size 104): 763130492d46c2ab4cc29c7bc2b0efbfbd78c29b29c3ab1e4723dea6efbfbdefbfbdefbfbdefbfbd66c2a275efbfbdefbfbdc4a6 Original value in hex size(68): 763130492d46ab4c9c7bb0e9789b29c3ab1e4723dea6c8e5d2f9b266a275e1fbe4a6 there are some data that is parsed correct but time to time it is broken On Mon, Nov 28, 2022 at 11:17 AM Tony Rietwyk wrote: Hi Yauheni, You don't say which type of database you are using? There may be issues with the database client library altering the blob to UTF8. Though I would expect the python code using the same client to have the same problem. Assuming that QVariant has the byte array unaltered, then QVariant.toByteArray().length() should give 34. Using qstrlen, suggests that you are using QVariant.toString instead. To print the hex you should just use QVariant.toByteArray().toHex(). Hope that helps, Tony On 28/11/2022 6:46 pm, Yauheni Pervenenka via Interest wrote: > Hello, community > I have faced with issue that qvariant using qstrlen for getting data > size, but sometimes it returns wrong size especially for BLOB data(if > there are some special symbols), are there any way to get data correct? > > Note: > Database value 34, python returns correct data size, but > qvariant::toByteArray returns 52 > > Python data and Database data in hex(size > 68): 763130492d46ab4c9c7bb0e9789b29c3ab1e4723dea6c8e5d2f9b266a275e1fbe4a6 > > Qt data in hex(size 104): > 763130492d46c2ab4cc29c7bc2b0efbfbd78c29b29c3ab1e4723dea6efbfbdefbfbdefbfbdefbfbd66c2a275efbfbdefbfbdc4a6 > Hi, most of the data is preserved in Qt but some other data is injected midstream. Just a guess, but maybe the Qt version has a multithreading issue, like two threads are settings values in the same array at the same time. ___ Interest mailing list Interest@qt-project.org https://lists.qt-project.org/listinfo/interest
Re: [Interest] Qt Sql BLOB data converting
Hi, Tony, thanks for your reply, OS: win 11 DB: SQLite Unfortunately, QVariant::toByteArray returns size 52(hex 104), QVariant::toString returns size 30 There is data returned by QVariant::toByteArray::toHex Qt data in hex(size 104): 763130492d46c2ab4cc29c7bc2b0efbfbd78c29b29c3ab1e4723dea6efbfbdefbfbdefbfbdefbfbd66c2a275efbfbdefbfbdc4a6 Original value in hex size(68): 763130492d46ab4c9c7bb0e9789b29c3ab1e4723dea6c8e5d2f9b266a275e1fbe4a6 there are some data that is parsed correct but time to time it is broken On Mon, Nov 28, 2022 at 11:17 AM Tony Rietwyk wrote: > Hi Yauheni, > > You don't say which type of database you are using? There may be issues > with the database client library altering the blob to UTF8. Though I > would expect the python code using the same client to have the same > problem. > > Assuming that QVariant has the byte array unaltered, then > QVariant.toByteArray().length() should give 34. Using qstrlen, suggests > that you are using QVariant.toString instead. To print the hex you > should just use QVariant.toByteArray().toHex(). > > Hope that helps, > > Tony > > > On 28/11/2022 6:46 pm, Yauheni Pervenenka via Interest wrote: > > Hello, community > > I have faced with issue that qvariant using qstrlen for getting data > > size, but sometimes it returns wrong size especially for BLOB data(if > > there are some special symbols), are there any way to get data correct? > > > > Note: > > Database value 34, python returns correct data size, but > > qvariant::toByteArray returns 52 > > > > Python data and Database data in hex(size > > 68): 763130492d46ab4c9c7bb0e9789b29c3ab1e4723dea6c8e5d2f9b266a275e1fbe4a6 > > > > Qt data in hex(size 104): > > > 763130492d46c2ab4cc29c7bc2b0efbfbd78c29b29c3ab1e4723dea6efbfbdefbfbdefbfbdefbfbd66c2a275efbfbdefbfbdc4a6 > > > > ___ > > Interest mailing list > > Interest@qt-project.org > > https://lists.qt-project.org/listinfo/interest > ___ > Interest mailing list > Interest@qt-project.org > https://lists.qt-project.org/listinfo/interest > ___ Interest mailing list Interest@qt-project.org https://lists.qt-project.org/listinfo/interest
Re: [Interest] Qt Sql BLOB data converting
Hi Yauheni, You don't say which type of database you are using? There may be issues with the database client library altering the blob to UTF8. Though I would expect the python code using the same client to have the same problem. Assuming that QVariant has the byte array unaltered, then QVariant.toByteArray().length() should give 34. Using qstrlen, suggests that you are using QVariant.toString instead. To print the hex you should just use QVariant.toByteArray().toHex(). Hope that helps, Tony On 28/11/2022 6:46 pm, Yauheni Pervenenka via Interest wrote: Hello, community I have faced with issue that qvariant using qstrlen for getting data size, but sometimes it returns wrong size especially for BLOB data(if there are some special symbols), are there any way to get data correct? Note: Database value 34, python returns correct data size, but qvariant::toByteArray returns 52 Python data and Database data in hex(size 68): 763130492d46ab4c9c7bb0e9789b29c3ab1e4723dea6c8e5d2f9b266a275e1fbe4a6 Qt data in hex(size 104): 763130492d46c2ab4cc29c7bc2b0efbfbd78c29b29c3ab1e4723dea6efbfbdefbfbdefbfbdefbfbd66c2a275efbfbdefbfbdc4a6 ___ Interest mailing list Interest@qt-project.org https://lists.qt-project.org/listinfo/interest ___ Interest mailing list Interest@qt-project.org https://lists.qt-project.org/listinfo/interest
