Re: [I] [Support] Request: Ability to Invoke Scala Code for Operations in oneTable [incubator-xtable]

2024-03-10 Thread via GitHub
soumilshah1995 commented on issue #353: URL: https://github.com/apache/incubator-xtable/issues/353#issuecomment-1987227772 Thank you very much. I wholeheartedly agree that addressing this issue in the future is crucial. From my testing, it appears that the functionality of onetable is

Re: [I] [Support] Request: Ability to Invoke Scala Code for Operations in oneTable [incubator-xtable]

2024-03-09 Thread via GitHub
the-other-tim-brown commented on issue #353: URL: https://github.com/apache/incubator-xtable/issues/353#issuecomment-1987022944 The "parent pom" as it is sometimes referred is where we keep all the version information: https://github.com/apache/incubator-xtable/blob/main/pom.xml#L29

Re: [I] [Support] Request: Ability to Invoke Scala Code for Operations in oneTable [incubator-xtable]

2024-03-09 Thread via GitHub
soumilshah1995 commented on issue #353: URL: https://github.com/apache/incubator-xtable/issues/353#issuecomment-1986995585 ![Screenshot 2024-03-09 at 5 07 30  PM](https://github.com/apache/incubator-xtable/assets/39345855/e794708a-6988-4f0a-9416-dc876593d689) Tutorials are

Re: [I] [Support] Request: Ability to Invoke Scala Code for Operations in oneTable [incubator-xtable]

2024-03-09 Thread via GitHub
soumilshah1995 commented on issue #353: URL: https://github.com/apache/incubator-xtable/issues/353#issuecomment-1986991321 ![Screenshot 2024-03-09 at 4 58 38  PM](https://github.com/apache/incubator-xtable/assets/39345855/29bea16e-f7ab-4316-8855-14148beb9aec) Test passed for

Re: [I] [Support] Request: Ability to Invoke Scala Code for Operations in oneTable [incubator-xtable]

2024-03-09 Thread via GitHub
soumilshah1995 commented on issue #353: URL: https://github.com/apache/incubator-xtable/issues/353#issuecomment-1986990165 is this a bug then for delta not sure why it failed for delta ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on

Re: [I] [Support] Request: Ability to Invoke Scala Code for Operations in oneTable [incubator-xtable]

2024-03-09 Thread via GitHub
soumilshah1995 commented on issue #353: URL: https://github.com/apache/incubator-xtable/issues/353#issuecomment-1986990062 Guess what it works for ICEBERG failed for Delta -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

Re: [I] [Support] Request: Ability to Invoke Scala Code for Operations in oneTable [incubator-xtable]

2024-03-09 Thread via GitHub
soumilshah1995 commented on issue #353: URL: https://github.com/apache/incubator-xtable/issues/353#issuecomment-1986988856 where do I need to change pom.xml I mean which directory

Re: [I] [Support] Request: Ability to Invoke Scala Code for Operations in oneTable [incubator-xtable]

2024-03-09 Thread via GitHub
the-other-tim-brown commented on issue #353: URL: https://github.com/apache/incubator-xtable/issues/353#issuecomment-1986938917 You can change the versions used in the `pom.xml` and rebuild the jar. You will likely need to use a Delta 2.3.X version based on this table:

Re: [I] [Support] Request: Ability to Invoke Scala Code for Operations in oneTable [incubator-xtable]

2024-03-08 Thread via GitHub
soumilshah1995 commented on issue #353: URL: https://github.com/apache/incubator-xtable/issues/353#issuecomment-1986271512 didn't work by the way how would I build for specific spark version @the-other-tim-brown or @sagarlakshmipathy -- This is an automated message from the

Re: [I] [Support] Request: Ability to Invoke Scala Code for Operations in oneTable [incubator-xtable]

2024-03-08 Thread via GitHub
soumilshah1995 commented on issue #353: URL: https://github.com/apache/incubator-xtable/issues/353#issuecomment-1986202928 let me try this today not sure if works I will clone glue 4 container locally clone onetable project and try to build jar file inside container lets see how that

Re: [I] [Support] Request: Ability to Invoke Scala Code for Operations in oneTable [incubator-xtable]

2024-03-08 Thread via GitHub
soumilshah1995 commented on issue #353: URL: https://github.com/apache/incubator-xtable/issues/353#issuecomment-1985722411 I've encountered a slight challenge with our AWS setup. While AWS utilizes Spark 3.3-amzn, my local environment runs Spark 3.4. My jar files are built for Spark 3.4.

Re: [I] [Support] Request: Ability to Invoke Scala Code for Operations in oneTable [incubator-xtable]

2024-03-07 Thread via GitHub
the-other-tim-brown commented on issue #353: URL: https://github.com/apache/incubator-xtable/issues/353#issuecomment-1985052827 This ticket is fine. There is no NullPointerException thrown here. If you inspect the stacktrace you'll see a `ClassNotFoundException`. This means that there is

Re: [I] [Support] Request: Ability to Invoke Scala Code for Operations in oneTable [incubator-xtable]

2024-03-07 Thread via GitHub
soumilshah1995 commented on issue #353: URL: https://github.com/apache/incubator-xtable/issues/353#issuecomment-1984799106 For Glue Scala should I create separate Ticket or this thread is good ? -- This is an automated message from the Apache Git Service. To respond to the message,

Re: [I] [Support] Request: Ability to Invoke Scala Code for Operations in oneTable [incubator-xtable]

2024-03-07 Thread via GitHub
soumilshah1995 commented on issue #353: URL: https://github.com/apache/incubator-xtable/issues/353#issuecomment-1983479883 I think as sagar was saying there is NULL pointer exception hmm -- This is an automated message from the Apache Git Service. To respond to the message, please log on

Re: [I] [Support] Request: Ability to Invoke Scala Code for Operations in oneTable [incubator-xtable]

2024-03-07 Thread via GitHub
soumilshah1995 commented on issue #353: URL: https://github.com/apache/incubator-xtable/issues/353#issuecomment-1983449032 Hey @the-other-tim-brown I started some work with glue and onetable on delta streamer here is what I am doing and I know 99% its jar issue # Step 1:

Re: [I] [Support] Request: Ability to Invoke Scala Code for Operations in oneTable [incubator-xtable]

2024-03-06 Thread via GitHub
soumilshah1995 commented on issue #353: URL: https://github.com/apache/incubator-xtable/issues/353#issuecomment-1980942556 Hello @the-other-tim-brown I'm curious about the Python wrapper 253 Ticket. Do you think it would enable us to utilize Onetable in PySpark? I haven't

Re: [I] [Support] Request: Ability to Invoke Scala Code for Operations in oneTable [incubator-xtable]

2024-03-04 Thread via GitHub
the-other-tim-brown commented on issue #353: URL: https://github.com/apache/incubator-xtable/issues/353#issuecomment-1977142750 I have not worked with Glue before. Can you help me understand where it differs from regular Scala code? There is no support for python at this time but

Re: [I] [Support] Request: Ability to Invoke Scala Code for Operations in oneTable [incubator-xtable]

2024-03-04 Thread via GitHub
soumilshah1995 commented on issue #353: URL: https://github.com/apache/incubator-xtable/issues/353#issuecomment-1976608176 Hello @the-other-tim-brown, Thank you for your response. I understand that we can add Jar files, but I was wondering if you have any examples of glue code with

Re: [I] [Support] Request: Ability to Invoke Scala Code for Operations in oneTable [incubator-xtable]

2024-03-03 Thread via GitHub
the-other-tim-brown commented on issue #353: URL: https://github.com/apache/incubator-xtable/issues/353#issuecomment-1975733480 I still don't really understand the request. There is functionality to add jars to your glue jobs with `--extra-jars`

Re: [I] [Support] Request: Ability to Invoke Scala Code for Operations in oneTable [incubator-xtable]

2024-02-29 Thread via GitHub
soumilshah1995 commented on issue #353: URL: https://github.com/apache/incubator-xtable/issues/353#issuecomment-1972127195 I am interested in running oneTable on Glue for my synchronization process. Specifically, I am looking for a method to translate metadata efficiently. I was hoping

Re: [I] [Support] Request: Ability to Invoke Scala Code for Operations in oneTable [incubator-xtable]

2024-02-29 Thread via GitHub
the-other-tim-brown commented on issue #353: URL: https://github.com/apache/incubator-xtable/issues/353#issuecomment-1972069848 Do you mean you want to call the OneTable classes directly from scala? I've done something similar locally in the docker demo with the notebook but will need to

[I] [Support] Request: Ability to Invoke Scala Code for Operations in oneTable [incubator-xtable]

2024-02-29 Thread via GitHub
soumilshah1995 opened a new issue, #353: URL: https://github.com/apache/incubator-xtable/issues/353 Hello, I hope this message finds you well. I'm currently in the midst of experimenting with oneTable and conducting some proof of concept work. As I delve deeper into this, I've