Re: [sqlalchemy] Modifying Query Object

[Luis Del Rio IV]

 The sql query itself returns several rows, as it should. But when trying
to combine the max using sqlalchemy the rows return as the following.

Received incompatible instance \"(, '2021-04-10 18', Decimal('7294.0'))\".",

Here I am able to get the max for that row group, but am unable to pass it
into the DataModel object


Luis Del Rio IV
DevOps Engineer
[image: Colovore]
1101 Space Park Drive
Santa Clara, CA 95054
<https://www.google.com/maps/place/1101+Space+Park+Dr,+Santa+Clara,+CA+95054/@37.3766977,-121.9552767,17z/data=!3m1!4b1!4m5!3m4!1s0x808fca298ada9ab5:0x8b082f7c3541c8bf!8m2!3d37.3766977!4d-121.953088>
mobile: 408. <%204085315362>*375.4195*
24/7: 408.330.9290 <%20408%203309290>


On Tue, Apr 11, 2023 at 4:59 PM S Mahabl  wrote:

> Do you get many rows?
>
> SELECT  date_format(data.timestamp, "%Y-%m-%d %H"), max(data.value)  AS
> data_value
> from data
> GROUP BY date_format(data.timestamp, "%Y-%m-%d %H")
>
> On Tue, Apr 11, 2023 at 4:24 PM Luis Del Rio IV  wrote:
>
>> Hello,
>>
>> I am currently using the following sqlalchemy code,
>>
>> _query = super().get_query(model, info, sort, **args)
>> query = _query.group_by(
>> func.date_format(DataModel.timestamp, "%Y-%m-%d %H")
>> )
>> return query
>>
>> I am trying to aggregate the the max value of a field from
>> DataModel.value utilizing the group by clause.
>>
>> In simple sql, you would do the following.
>>
>> SELECT  max(data.value)  AS data_value
>> from data
>> GROUP BY date_format(data.timestamp, "%Y-%m-%d %H")
>>
>> What would the proper way to express this? The various methods I have
>> tried somehow overwrite the original query and do not map to our attributes.
>>
>> --
>> SQLAlchemy -
>> The Python SQL Toolkit and Object Relational Mapper
>>
>> http://www.sqlalchemy.org/
>>
>> To post example code, please provide an MCVE: Minimal, Complete, and
>> Verifiable Example. See http://stackoverflow.com/help/mcve for a full
>> description.
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "sqlalchemy" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to sqlalchemy+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/sqlalchemy/a73c1830-8c64-437a-8ea7-a171767e2223n%40googlegroups.com
>> <https://groups.google.com/d/msgid/sqlalchemy/a73c1830-8c64-437a-8ea7-a171767e2223n%40googlegroups.com?utm_medium=email_source=footer>
>> .
>>
> --
> SQLAlchemy -
> The Python SQL Toolkit and Object Relational Mapper
>
> http://www.sqlalchemy.org/
>
> To post example code, please provide an MCVE: Minimal, Complete, and
> Verifiable Example. See http://stackoverflow.com/help/mcve for a full
> description.
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "sqlalchemy" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/sqlalchemy/j5fIV6NmAns/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> sqlalchemy+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/sqlalchemy/CAOV%2B3C2PU2ndh9Uf-ZGtDj_ao-3rhQATE9MjYAppSSnwKT6%2Beg%40mail.gmail.com
> <https://groups.google.com/d/msgid/sqlalchemy/CAOV%2B3C2PU2ndh9Uf-ZGtDj_ao-3rhQATE9MjYAppSSnwKT6%2Beg%40mail.gmail.com?utm_medium=email_source=footer>
> .
>

-- 
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sqlalchemy/CABKymD8cF3JQoxD_-qH6Y4kg-tp9GOGM2kO3bqrm_qY6Ogz%3DcQ%40mail.gmail.com.

Re: [sqlalchemy] Modifying Query Object

[Luis Del Rio IV]

 The sql query itself returns several rows, as it should. But when trying 
to combine the max using sqlalchemy the rows return as the following.

Received incompatible instance "(server.db.models.Data object at , 
'2021-04-10 18', Decimal('7294.0'))".", 

Here I am able to get the max for that row group, but am unable to pass it 
into the DataModel object


On Tuesday, April 11, 2023 at 4:59:15 PM UTC-7 S Mahabl wrote:

> Do you get many rows?
>
> SELECT  date_format(data.timestamp, "%Y-%m-%d %H"), max(data.value)  AS 
> data_value
> from data
> GROUP BY date_format(data.timestamp, "%Y-%m-%d %H")
> On Tue, Apr 11, 2023 at 4:24 PM Luis Del Rio IV  
> wrote:
>
>> Hello,
>>
>> I am currently using the following sqlalchemy code,
>>
>> _query = super().get_query(model, info, sort, **args)
>> query = _query.group_by(
>> func.date_format(DataModel.timestamp, "%Y-%m-%d %H")
>> )
>> return query
>>
>> I am trying to aggregate the the max value of a field from 
>> DataModel.value utilizing the group by clause.
>>
>> In simple sql, you would do the following.
>>
>> SELECT  max(data.value)  AS data_value
>> from data
>> GROUP BY date_format(data.timestamp, "%Y-%m-%d %H")
>>
>> What would the proper way to express this? The various methods I have 
>> tried somehow overwrite the original query and do not map to our attributes.
>>
>> -- 
>> SQLAlchemy - 
>> The Python SQL Toolkit and Object Relational Mapper
>>  
>> http://www.sqlalchemy.org/
>>  
>> To post example code, please provide an MCVE: Minimal, Complete, and 
>> Verifiable Example. See http://stackoverflow.com/help/mcve for a full 
>> description.
>> --- 
>> You received this message because you are subscribed to the Google Groups 
>> "sqlalchemy" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to sqlalchemy+...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/sqlalchemy/a73c1830-8c64-437a-8ea7-a171767e2223n%40googlegroups.com
>>  
>> <https://groups.google.com/d/msgid/sqlalchemy/a73c1830-8c64-437a-8ea7-a171767e2223n%40googlegroups.com?utm_medium=email_source=footer>
>> .
>>
>

-- 
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sqlalchemy/dc2df88c-b30d-4cb2-a017-385edc46e1cfn%40googlegroups.com.

Re: [sqlalchemy] Modifying Query Object

[Luis Del Rio IV]

The sql query itself returns several rows, as it should. But when trying to 
combine the max using sqlalchemy the rows return as the following.

Received incompatible instance \"(, '2021-04-10 18', Decimal('7294.0'))\".", 

Here I am able to get the max for that row group, but am unable to pass it 
into the DataModel object
On Tuesday, April 11, 2023 at 4:59:15 PM UTC-7 S Mahabl wrote:

> Do you get many rows?
>
> SELECT  date_format(data.timestamp, "%Y-%m-%d %H"), max(data.value)  AS 
> data_value
> from data
> GROUP BY date_format(data.timestamp, "%Y-%m-%d %H")
> On Tue, Apr 11, 2023 at 4:24 PM Luis Del Rio IV  
> wrote:
>
>> Hello,
>>
>> I am currently using the following sqlalchemy code,
>>
>> _query = super().get_query(model, info, sort, **args)
>> query = _query.group_by(
>> func.date_format(DataModel.timestamp, "%Y-%m-%d %H")
>> )
>> return query
>>
>> I am trying to aggregate the the max value of a field from 
>> DataModel.value utilizing the group by clause.
>>
>> In simple sql, you would do the following.
>>
>> SELECT  max(data.value)  AS data_value
>> from data
>> GROUP BY date_format(data.timestamp, "%Y-%m-%d %H")
>>
>> What would the proper way to express this? The various methods I have 
>> tried somehow overwrite the original query and do not map to our attributes.
>>
>> -- 
>> SQLAlchemy - 
>> The Python SQL Toolkit and Object Relational Mapper
>>  
>> http://www.sqlalchemy.org/
>>  
>> To post example code, please provide an MCVE: Minimal, Complete, and 
>> Verifiable Example. See http://stackoverflow.com/help/mcve for a full 
>> description.
>> --- 
>> You received this message because you are subscribed to the Google Groups 
>> "sqlalchemy" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to sqlalchemy+...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/sqlalchemy/a73c1830-8c64-437a-8ea7-a171767e2223n%40googlegroups.com
>>  
>> <https://groups.google.com/d/msgid/sqlalchemy/a73c1830-8c64-437a-8ea7-a171767e2223n%40googlegroups.com?utm_medium=email_source=footer>
>> .
>>
>

-- 
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sqlalchemy/94139311-fe93-4584-b07a-239ec3b85159n%40googlegroups.com.

Re: [sqlalchemy] Modifying Query Object

[Luis Del Rio IV]

 The sql query itself returns several rows, as it should. But when trying 
to combine the max using sqlalchemy the rows return as the following.

Received incompatible instance \"(, '2021-04-10 18', Decimal('7294.0'))\".", 

Here I am able to get the max for that row group, but am unable to pass it 
into the DataModel object

On Tuesday, April 11, 2023 at 4:59:15 PM UTC-7 S Mahabl wrote:

> Do you get many rows?
>
> SELECT  date_format(data.timestamp, "%Y-%m-%d %H"), max(data.value)  AS 
> data_value
> from data
> GROUP BY date_format(data.timestamp, "%Y-%m-%d %H")
> On Tue, Apr 11, 2023 at 4:24 PM Luis Del Rio IV  
> wrote:
>
>> Hello,
>>
>> I am currently using the following sqlalchemy code,
>>
>> _query = super().get_query(model, info, sort, **args)
>> query = _query.group_by(
>> func.date_format(DataModel.timestamp, "%Y-%m-%d %H")
>> )
>> return query
>>
>> I am trying to aggregate the the max value of a field from 
>> DataModel.value utilizing the group by clause.
>>
>> In simple sql, you would do the following.
>>
>> SELECT  max(data.value)  AS data_value
>> from data
>> GROUP BY date_format(data.timestamp, "%Y-%m-%d %H")
>>
>> What would the proper way to express this? The various methods I have 
>> tried somehow overwrite the original query and do not map to our attributes.
>>
>> -- 
>> SQLAlchemy - 
>> The Python SQL Toolkit and Object Relational Mapper
>>  
>> http://www.sqlalchemy.org/
>>  
>> To post example code, please provide an MCVE: Minimal, Complete, and 
>> Verifiable Example. See http://stackoverflow.com/help/mcve for a full 
>> description.
>> --- 
>> You received this message because you are subscribed to the Google Groups 
>> "sqlalchemy" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to sqlalchemy+...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/sqlalchemy/a73c1830-8c64-437a-8ea7-a171767e2223n%40googlegroups.com
>>  
>> <https://groups.google.com/d/msgid/sqlalchemy/a73c1830-8c64-437a-8ea7-a171767e2223n%40googlegroups.com?utm_medium=email_source=footer>
>> .
>>
>

-- 
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sqlalchemy/3eeda16b-9c36-4ca5-91b5-cba325343fban%40googlegroups.com.

Re: [sqlalchemy] Query object modification

[Luis Del Rio IV]

Mike,

Would this select query be able to get our aggregated data?

query = select(func.max(DataModel.value)).select_from(_query
.subquery()).group_by(
func.date_format(DataModel.timestamp, "%Y-%m-%d %H")
)

We tried this route but are now getting this error

  "errors": [
{
  "message": "'Select' object has no attribute 'statement'",
  "locations": [
{
  "line": 2,
  "column": 3
}
  ],

On Thursday, April 13, 2023 at 10:13:44 AM UTC-7 Peter Harrison wrote:

> Thanks Mike,
>
> Ideally we'd prefer to find a solution via Graphene-SQLAlchemy. 
> Unfortunately we don't have the luxury of creating our own query when 
> interacting with Graphene-SQLAlchemy.
>
> So the key question for us is, can you modify an existing 
> sqlalchemy.orm.query.Query object to insert a "func.max(Data.value)" object 
> into the select? This is what Graphene-SQLAlchemy is giving us.
> If this is possible, then the group_by part is easy. We have tested that 
> frequently, the hard part is the modifying the original "select" object.
>
> 1. We have tried add_columns, but that adds in incompatible object type in 
> the GraphQL results making it an unusable option.
> 2. We thought that modifying the select would be  possible using data with 
> "statement.froms" but can't figure out how to update the MetaData object in 
> it
>
> If modifying the "select" after its creation is not possible, we need to 
> start considering using a separate reporting table with hourly data.
>
> On Wednesday, April 12, 2023 at 3:12:46 PM UTC-7 Mike Bayer wrote:
>
>>
>>
>> On Wed, Apr 12, 2023, at 5:21 PM, Luis Del Rio IV wrote:
>>
>> I am currently using the following sqlalchemy code,
>>
>> _query = super().get_query(model, info, sort, **args)
>> query = _query.group_by(
>> func.date_format(DataModel.timestamp, "%Y-%m-%d %H")
>> )
>> return query
>>
>> I am trying to aggregate the the max value of a field from 
>> DataModel.value utilizing the group by clause.
>>
>> In simple sql, you would do the following.
>>
>> SELECT  max(data.value)  AS data_value
>> from data
>> GROUP BY date_format(data.timestamp, "%Y-%m-%d %H")
>>
>> What would the proper way to express this? The various methods I have 
>> tried somehow overwrite the original query and do not map to our attributes.
>>
>>
>> using legacy query style:
>>
>> q1 = session.query(func.max(Data.value)).group_by(
>> func.date_format(Data.timestamp, "%Y-%m-%d %H")
>> )
>>
>>
>> using 2.0 style select(), replace "session.query" with "select":
>>
>> s1 = select(func.max(Data.value)).group_by(
>> func.date_format(Data.timestamp, "%Y-%m-%d %H")
>> )
>>
>> POC script is at the bottom of this email.
>>
>>
>>
>> I have some additional context on the issue
>>
>>1. We are using the graphene-sqlalchemy package.
>>2. When you do a GraphQL web api query, the package creates a 
>> sqlalchemy.orm.query.Query 
>>object.
>>3. We want to modify this standard query that the package creates so 
>>that we can do the group_by action to help us get the maximum time series 
>>value for every hour because this is not possible using GraphQL.
>>4. Talking to the graphene-sqlalchemy team they told us the best 
>>place to do the modification to the standardized query is in the 
>> get_query 
>>method (line 67) in this file: https://github.com/graphql-
>>python/graphene-sqlalchemy/blob/master/graphene_sqlalchemy/fields.py 
>>
>> <https://github.com/graphql-python/graphene-sqlalchemy/blob/master/graphene_sqlalchemy/fields.py>
>>5. The standardized query we need to modify translates to this SQL 
>>statement: 
>>
>> SELECT sy_data.oid_id, sy_data.rrd_timestamp, sy_data.cabinet_id, sy_data
>> .customer_id, sy_data.value, sy_data.active_flag, sy_data.timestamp 
>> FROM sy_data 
>> WHERE sy_data.rrd_timestamp >= %(rrd_timestamp_1)s AND 
>> sy_data.rrd_timestamp <= %(rrd_timestamp_2)s AND (sy_data.oid_id = 
>> %(oid_id_1)s OR sy_data.oid_id = %(o
>> id_id_2)s) ORDER BY sy_data.oid_id ASC, sy_data.rrd_timestamp ASC
>>
>> Therefore we need to find a way to insert a func.max for the values in 
>> the first part of the SELECT statement, (before the FROM). It is easy for 
>> us to apend the group_b

[sqlalchemy] Query object modification

[Luis Del Rio IV]

 I am currently using the following sqlalchemy code,

_query = super().get_query(model, info, sort, **args)
query = _query.group_by(
func.date_format(DataModel.timestamp, "%Y-%m-%d %H")
)
return query

I am trying to aggregate the the max value of a field from DataModel.value 
utilizing the group by clause.

In simple sql, you would do the following.

SELECT  max(data.value)  AS data_value
from data
GROUP BY date_format(data.timestamp, "%Y-%m-%d %H")

What would the proper way to express this? The various methods I have tried 
somehow overwrite the original query and do not map to our attributes.

I have some additional context on the issue

   1. We are using the graphene-sqlalchemy package.
   2. When you do a GraphQL web api query, the package creates a 
sqlalchemy.orm.query.Query 
   object.
   3. We want to modify this standard query that the package creates so 
   that we can do the group_by action to help us get the maximum time series 
   value for every hour because this is not possible using GraphQL.
   4. Talking to the graphene-sqlalchemy team they told us the best place 
   to do the modification to the standardized query is in the get_query method 
   (line 67) in this file: https://github.com/graphql-
   python/graphene-sqlalchemy/blob/master/graphene_sqlalchemy/fields.py 
   

   5. The standardized query we need to modify translates to this SQL 
   statement: 

SELECT sy_data.oid_id, sy_data.rrd_timestamp, sy_data.cabinet_id, sy_data
.customer_id, sy_data.value, sy_data.active_flag, sy_data.timestamp 
FROM sy_data 
WHERE sy_data.rrd_timestamp >= %(rrd_timestamp_1)s AND 
sy_data.rrd_timestamp <= %(rrd_timestamp_2)s AND (sy_data.oid_id = 
%(oid_id_1)s OR sy_data.oid_id = %(o
id_id_2)s) ORDER BY sy_data.oid_id ASC, sy_data.rrd_timestamp ASC

Therefore we need to find a way to insert a func.max for the values in the 
first part of the SELECT statement, (before the FROM). It is easy for us to 
apend the group_by like this.

query.group_by(
func.date_format(DataModel.timestamp, "%Y-%m-%d %H")
)

The big issue for us is to figure out how to insert the func.max

Getting a solution to this will help the graphene-sqlalchemy team create 
better documentation.

-- 
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sqlalchemy/742e96bc-f6cc-43ca-8a2e-697b3803301cn%40googlegroups.com.

[sqlalchemy] Modifying Query Object

[Luis Del Rio IV]

Hello,

I am currently using the following sqlalchemy code,

_query = super().get_query(model, info, sort, **args)
query = _query.group_by(
func.date_format(DataModel.timestamp, "%Y-%m-%d %H")
)
return query

I am trying to aggregate the the max value of a field from DataModel.value 
utilizing the group by clause.

In simple sql, you would do the following.

SELECT  max(data.value)  AS data_value
from data
GROUP BY date_format(data.timestamp, "%Y-%m-%d %H")

What would the proper way to express this? The various methods I have tried 
somehow overwrite the original query and do not map to our attributes.

-- 
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sqlalchemy/a73c1830-8c64-437a-8ea7-a171767e2223n%40googlegroups.com.