[I] DatabricksSQLOperator struggles with parsing the data [airflow]

2024-01-17 Thread via GitHub
aru-trackunit opened a new issue, #36838: URL: https://github.com/apache/airflow/issues/36838 ### Apache Airflow Provider(s) databricks ### Versions of Apache Airflow Providers The error has not been present in version `apache-airflow-providers-databricks==4.7.0` I up

[I] DatabricksSQLOperator struggles with parsing the data [airflow]

2024-01-17 Thread via GitHub
aru-trackunit opened a new issue, #36839: URL: https://github.com/apache/airflow/issues/36839 ### Apache Airflow Provider(s) databricks ### Versions of Apache Airflow Providers The error has not been present in version apache-airflow-providers-databricks==4.7.0 I upgr

Re: [I] DatabricksSQLOperator struggles with parsing the data [airflow]

2024-03-25 Thread via GitHub
github-actions[bot] commented on issue #36838: URL: https://github.com/apache/airflow/issues/36838#issuecomment-2019139299 This issue has been automatically marked as stale because it has been open for 14 days with no response from the author. It will be closed in next 7 days if no further

Re: [I] DatabricksSQLOperator struggles with parsing the data [airflow]

2024-03-26 Thread via GitHub
aru-trackunit commented on issue #36838: URL: https://github.com/apache/airflow/issues/36838#issuecomment-2019858056 I think it's related to this issue https://github.com/databricks/databricks-sql-python/issues/326 -- This is an automated message from the Apache Git Service. To respond t

Re: [I] DatabricksSQLOperator struggles with parsing the data [airflow]

2024-04-11 Thread via GitHub
github-actions[bot] commented on issue #36838: URL: https://github.com/apache/airflow/issues/36838#issuecomment-2050747458 This issue has been automatically marked as stale because it has been open for 14 days with no response from the author. It will be closed in next 7 days if no further

Re: [I] DatabricksSQLOperator struggles with parsing the data [airflow]

2024-02-18 Thread via GitHub
bolkedebruin closed issue #36839: DatabricksSQLOperator struggles with parsing the data URL: https://github.com/apache/airflow/issues/36839 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specif

Re: [I] DatabricksSQLOperator struggles with parsing the data [airflow]

2024-02-21 Thread via GitHub
w0ut0 commented on issue #36838: URL: https://github.com/apache/airflow/issues/36838#issuecomment-1956088214 We have the same issue on Airflow 2.7.1 and 2.8.1, and both on provider 6.1.0 and 6.2.0. Upgrading pandas to 2.1.4 did not solve the issue for us. Downgrading the provider to 5.

Re: [I] DatabricksSQLOperator struggles with parsing the data [airflow]

2024-03-09 Thread via GitHub
potiuk commented on issue #36838: URL: https://github.com/apache/airflow/issues/36838#issuecomment-1986914063 @aru-trackunit - seems that @Joffreybvn cannot reproduce the issue - can you please provide a bit more information as he asked in https://github.com/apache/airflow/issues/36838#issu

Re: [I] DatabricksSQLOperator struggles with parsing the data [airflow]

2024-03-11 Thread via GitHub
aru-trackunit commented on issue #36838: URL: https://github.com/apache/airflow/issues/36838#issuecomment-1988736158 I discussed it with him on airflow's slack but it's a good idea to have a visibility over it in here as well. I have attached a `DatabricksSqlOperator` that is ready to tes

Re: [I] DatabricksSQLOperator struggles with parsing the data [airflow]

2024-01-17 Thread via GitHub
aru-trackunit commented on issue #36838: URL: https://github.com/apache/airflow/issues/36838#issuecomment-1895586081 There is also another ticket that is related to the similar code path: https://github.com/apache/airflow/issues/36839 -- This is an automated message from the Apache Git

Re: [I] DatabricksSQLOperator struggles with parsing the data [airflow]

2024-01-22 Thread via GitHub
bolkedebruin commented on issue #36838: URL: https://github.com/apache/airflow/issues/36838#issuecomment-1903744409 @Joffreybvn any idea? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the speci

Re: [I] DatabricksSQLOperator struggles with parsing the data [airflow]

2024-01-22 Thread via GitHub
Joffreybvn commented on issue #36839: URL: https://github.com/apache/airflow/issues/36839#issuecomment-1904100618 Indeed, a Databricks.Row.__fields__ is itself another Row object when the row is created in two steps. Fix for that is also coming in #36949 Thanks for the debug print ! --

Re: [I] DatabricksSQLOperator struggles with parsing the data [airflow]

2024-01-29 Thread via GitHub
aru-trackunit commented on issue #36839: URL: https://github.com/apache/airflow/issues/36839#issuecomment-1914628927 @Joffreybvn Just retested with `Databricks provider 6.1.0` on `airflow 2.8.1` and the issue still persists with different stacktrace: ``` [2024-01-29, 13:41:53 CET]

Re: [I] DatabricksSQLOperator struggles with parsing the data [airflow]

2024-01-29 Thread via GitHub
aru-trackunit commented on issue #36838: URL: https://github.com/apache/airflow/issues/36838#issuecomment-1914632249 @Joffreybvn Just retested with `Databricks provider 6.1.0` on `airflow 2.8.1` and the issue still persists with different stacktrace: ``` [2024-01-29, 13:41:39 CET]

Re: [I] DatabricksSQLOperator struggles with parsing the data [airflow]

2024-01-29 Thread via GitHub
Joffreybvn commented on issue #36839: URL: https://github.com/apache/airflow/issues/36839#issuecomment-1914955285 I could not reproduce your issue. But I found another bug where a single namedtuple 'Row' fails to be serialized, while a list of those namedtuples works fine. Is your query ret

Re: [I] DatabricksSQLOperator struggles with parsing the data [airflow]

2024-02-01 Thread via GitHub
odykstra commented on issue #36839: URL: https://github.com/apache/airflow/issues/36839#issuecomment-1921951126 Hi there. I'm running into exactly the same issue -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [I] DatabricksSQLOperator struggles with parsing the data [airflow]

2024-02-01 Thread via GitHub
odykstra commented on issue #36839: URL: https://github.com/apache/airflow/issues/36839#issuecomment-1922153837 Hi there. I'm running into this same issue. I'm running a merge statement that only returns one row with four columns. Appreciate your attention to this. Old version

Re: [I] DatabricksSQLOperator struggles with parsing the data [airflow]

2024-02-04 Thread via GitHub
Joffreybvn commented on issue #36838: URL: https://github.com/apache/airflow/issues/36838#issuecomment-1925889496 I cannot reproduce this error. Could you give me more info about your data ? Maybe a screenshot (like below) of what you have on the databrick's UI ? ![MicrosoftTeams-ima

Re: [I] DatabricksSQLOperator struggles with parsing the data [airflow]

2024-02-05 Thread via GitHub
amoralca16 commented on issue #36839: URL: https://github.com/apache/airflow/issues/36839#issuecomment-1926613645 Using pandas-2.1.4 solved the issue for me (Airflow 2.8.1) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and u

Re: [I] DatabricksSQLOperator struggles with parsing the data [airflow]

2024-02-05 Thread via GitHub
amoralca16 commented on issue #36838: URL: https://github.com/apache/airflow/issues/36838#issuecomment-1926615969 Using pandas-2.1.4 solved the issue for me (Airflow 2.8.1) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and u

Re: [I] DatabricksSQLOperator struggles with parsing the data [airflow]

2024-04-18 Thread via GitHub
github-actions[bot] closed issue #36838: DatabricksSQLOperator struggles with parsing the data URL: https://github.com/apache/airflow/issues/36838 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [I] DatabricksSQLOperator struggles with parsing the data [airflow]

2024-04-18 Thread via GitHub
github-actions[bot] commented on issue #36838: URL: https://github.com/apache/airflow/issues/36838#issuecomment-2065517415 This issue has been closed because it has not received response from the issue author. -- This is an automated message from the Apache Git Service. To respond to the