Re: [PR] GH-44800: [C#] Implement Flight SQL Client [arrow]

2025-07-21 Thread via GitHub 


CurtHagenlocher merged PR #44783:
URL: https://github.com/apache/arrow/pull/44783


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Re: [PR] GH-44800: [C#] Implement Flight SQL Client [arrow]

2025-07-21 Thread via GitHub 


CurtHagenlocher commented on code in PR #44783:
URL: https://github.com/apache/arrow/pull/44783#discussion_r2021649610


##
csharp/src/Apache.Arrow.Flight.Sql/Transaction.cs:
##
@@ -0,0 +1,48 @@
+// Licensed to the Apache Software Foundation (ASF) under one or more
+// contributor license agreements. See the NOTICE file distributed with
+// this work for additional information regarding copyright ownership.
+// The ASF licenses this file to You under the Apache License, Version 2.0
+// (the "License"); you may not use this file except in compliance with
+// the License.  You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+namespace Apache.Arrow.Flight.Sql;
+
+using Google.Protobuf; // Ensure you have the Protobuf dependency
+
+public readonly struct Transaction
+{
+private static readonly ByteString TransactionIdDefaultValue = 
ByteString.Empty;
+
+private readonly ByteString _transactionId;
+
+public ByteString TransactionId => _transactionId ?? 
TransactionIdDefaultValue;
+
+public static readonly Transaction NoTransaction = 
new(TransactionIdDefaultValue);
+
+public Transaction(ByteString transactionId)
+{
+_transactionId = ProtoPreconditions.CheckNotNull(transactionId, 
nameof(transactionId));
+}
+
+public Transaction(string transactionId)
+{
+_transactionId = ByteString.CopyFromUtf8(transactionId);
+}
+
+public bool IsValid() => _transactionId.Length > 0;

Review Comment:
   ```suggestion
   public bool IsValid => _transactionId.Length > 0;
   ```



##
csharp/src/Apache.Arrow.Flight.Sql/PreparedStatement.cs:
##
@@ -0,0 +1,383 @@
+// Licensed to the Apache Software Foundation (ASF) under one or more
+// contributor license agreements. See the NOTICE file distributed with
+// this work for additional information regarding copyright ownership.
+// The ASF licenses this file to You under the Apache License, Version 2.0
+// (the "License"); you may not use this file except in compliance with
+// the License.  You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+using System;
+using System.Collections.Generic;
+using System.Linq;
+using System.Text;
+using System.Threading;
+using System.Threading.Tasks;
+using Apache.Arrow.Flight.Sql.Client;
+using Arrow.Flight.Protocol.Sql;
+using Google.Protobuf;
+using Grpc.Core;
+
+namespace Apache.Arrow.Flight.Sql;
+
+public class PreparedStatement : IDisposable
+{
+private readonly FlightSqlClient _client;
+private readonly string _handle;
+private Schema _datasetSchema;
+private Schema _parameterSchema;

Review Comment:
   These are still unused as of the latest iteration. They're set by the 
constructor and never referenced again.



##
csharp/src/Apache.Arrow.Flight.Sql/FlightExtensions.cs:
##
@@ -0,0 +1,41 @@
+// Licensed to the Apache Software Foundation (ASF) under one or more
+// contributor license agreements. See the NOTICE file distributed with
+// this work for additional information regarding copyright ownership.
+// The ASF licenses this file to You under the Apache License, Version 2.0
+// (the "License"); you may not use this file except in compliance with
+// the License.  You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+using System;
+using Google.Protobuf;
+using Google.Protobuf.WellKnownTypes;
+
+namespace Apache.Arrow.Flight.Sql;
+
+internal static class FlightExtensions
+{
+public static byte[] PackAndSerialize(this IMessage command) => 
Any.Pack(command).ToByteArray();
+
+public static T ParseAndUnpack(this ByteString source) where T : 
IMessage, new() =>
+Any.Parser.ParseFrom(source).Unpack();
+
+public static int ExtractRowCount(this RecordBatch batch)

Review Comment:
   Why isn't this just `RecordBatch.Length`?



##
csharp/src/Apache.Arrow.Flight.Sql/DoPutResult.cs:
##

Re: [PR] GH-44800: [C#] Implement Flight SQL Client [arrow]

2025-07-21 Thread via GitHub 


HackPoint commented on PR #44783:
URL: https://github.com/apache/arrow/pull/44783#issuecomment-3096746066

   
   
   
   > One thing I didn't do was to resolve the question of the unused 
`_dataSetSchema` and `_parameterSchema` fields on `PreparedStatement`. Work is 
being done to populate these, but they are never used. I started removing them 
but then realized that I don't really understand the difference between these 
and the `GetSchemaAsync` call on the same class. Can you take another look and 
decide what you think should be done about this?
   
   After reviewing the C++ implementation (client.h / PreparedStatement), I 
confirmed that:
   ```cpp
   std::shared_ptr parameter_schema() const { return parameter_schema_; 
}
   std::shared_ptr dataset_schema() const { return dataset_schema_; }
   ```
   source: `[client.h / PreparedStatement]`
   
   These schemas are accessed directly by consumers after creating the prepared 
statement, allowing them to inspect the expected input/output without making 
additional calls.
   
   In contrast, `GetSchemaAsync()` in C# makes a separate server call to 
retrieve the schema again — which may be redundant if we already have it.
   
   So I made the design decision to expose both schemas as public, readonly 
properties:
   ```csharp
   public Schema DatasetSchema { get; }
   public Schema ParameterSchema { get; }
   ```
   This aligns with the C++ API and gives clients immediate access to the known 
schemas without having to re-fetch them. If schema refresh is ever required, 
they can still call `GetSchemaAsync()` manually.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Re: [PR] GH-44800: [C#] Implement Flight SQL Client [arrow]

2025-07-15 Thread via GitHub 


CurtHagenlocher commented on PR #44783:
URL: https://github.com/apache/arrow/pull/44783#issuecomment-3074093702

   @HackPoint See https://github.com/HackPoint/arrow/pull/2


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Re: [PR] GH-44800: [C#] Implement Flight SQL Client [arrow]

2025-07-14 Thread via GitHub 


HackPoint commented on PR #44783:
URL: https://github.com/apache/arrow/pull/44783#issuecomment-3070370780

   > @HackPoint, I do think it's important to add cancellation tokens to 
methods which themselves call other methods that already take a cancellation 
token -- particularly when the method in question is a public API. Even if no 
one ever ends up using it, the cost is quite small. By contrast, if one needs 
to be added later then we end up needing to override the method in order to 
preserve backwards binary compatibility.
   > 
   > I've gone through and figured out which methods fall into this category 
and published a version of this PR that includes those changes to 
https://github.com/CurtHagenlocher/arrow/tree/flight-sql-client-changes. I can 
also submit a PR from that branch to your branch if that helps. In addition to 
the cancellation token changes (which are the largest part of the delta), I've 
also implemented `IEquatable<>` on `Transaction` and `IAsyncDisposable` on 
`PreparedStatement`.
   > 
   > One thing I didn't do was to resolve the question of the unused 
`_dataSetSchema` and `_parameterSchema` fields on `PreparedStatement`. Work is 
being done to populate these, but they are never used. I started removing them 
but then realized that I don't really understand the difference between these 
and the `GetSchemaAsync` call on the same class. Can you take another look and 
decide what you think should be done about this?
   
   
   Hi Curt,
   
   Thanks a lot for pushing those updates—adding CancellationToken parameters 
and the extra interfaces makes perfect sense.
   Could you please go ahead and open a PR from flight-sql-client-changes to my 
branch? That will let me review and integrate your changes right away.
   I’ll take another pass over _dataSetSchema and _parameterSchema while I’m at 
it. My initial hunch is that _dataSetSchema may be redundant, but 
_parameterSchema is probably still necessary for optional parameter binding, so 
I’ll verify that and adjust accordingly.
   Once I’ve incorporated any tweaks you need, I’d appreciate it if you could 
merge my original PR as soon as everything looks good on your end.
   
   Thanks again for the thorough improvements!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Re: [PR] GH-44800: [C#] Implement Flight SQL Client [arrow]

2025-07-14 Thread via GitHub 


CurtHagenlocher commented on PR #44783:
URL: https://github.com/apache/arrow/pull/44783#issuecomment-3069382218

   @HackPoint, I do think it's important to add cancellation tokens to methods 
which themselves call other methods that already take a cancellation token -- 
particularly when the method in question is a public API. Even if no one ever 
ends up using it, the cost is quite small. By contrast, if one needs to be 
added later then we end up needing to override the method in order to preserve 
backwards binary compatibility.
   
   I've gone through and figured out which methods fall into this category and 
published a version of this PR that includes those changes to 
https://github.com/CurtHagenlocher/arrow/tree/flight-sql-client-changes. I can 
also submit a PR from that branch to your branch if that helps. In addition to 
the cancellation token changes (which are the largest part of the delta), I've 
also implemented `IEquatable<>` on `Transaction` and `IAsyncDisposable` on 
`PreparedStatement`.
   
   One thing I didn't do was to resolve the question of the unused 
`_dataSetSchema` and `_parameterSchema` fields on `PreparedStatement`. Work is 
being done to populate these, but they are never used. I started removing them 
but then realized that I don't really understand the difference between these 
and the `GetSchemaAsync` call on the same class. Can you take another look and 
decide what you think should be done about this?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Re: [PR] GH-44800: [C#] Implement Flight SQL Client [arrow]

2025-07-09 Thread via GitHub 


kou commented on PR #44783:
URL: https://github.com/apache/arrow/pull/44783#issuecomment-3054752366

   I'll move `csharp/` in apache/arrow to apache/arrow-dotnet after 
apache/arrow 21.0.0 release if nobody objects it: 
https://lists.apache.org/thread/j3v93j2s6wpr7hh0xxtofd5167nrf4g7


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Re: [PR] GH-44800: [C#] Implement Flight SQL Client [arrow]

2025-07-07 Thread via GitHub 


HackPoint commented on PR #44783:
URL: https://github.com/apache/arrow/pull/44783#issuecomment-3045691672

   > My current proposal is 
https://lists.apache.org/thread/y6zq26qb91b25lvs9cc7kb297lxdsjcl . (We move 
`csharp/` in apache/arrow to apache/arrow-dotnet BEFORE we merge this. We 
continue this in apache/arrow-dotnet.)
   
   @CurtHagenlocher Hi, still haven't been accepted yet, can you please assist 
with it?
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Re: [PR] GH-44800: [C#] Implement Flight SQL Client [arrow]

2025-05-28 Thread via GitHub 


kou commented on PR #44783:
URL: https://github.com/apache/arrow/pull/44783#issuecomment-2917689771

   My current proposal is 
https://lists.apache.org/thread/y6zq26qb91b25lvs9cc7kb297lxdsjcl .
   (We move `csharp/` in apache/arrow to apache/arrow-dotnet BEFORE we merge 
this. We continue this in apache/arrow-dotnet.)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Re: [PR] GH-44800: [C#] Implement Flight SQL Client [arrow]

2025-05-28 Thread via GitHub 


HackPoint commented on PR #44783:
URL: https://github.com/apache/arrow/pull/44783#issuecomment-2916887941

   @CurtHagenlocher @adamreeve @kou hey guys, take a look at this is it ready, 
is it good to go or should I do some changes?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Re: [PR] GH-44800: [C#] Implement Flight SQL Client [arrow]

2025-04-28 Thread via GitHub 


HackPoint commented on PR #44783:
URL: https://github.com/apache/arrow/pull/44783#issuecomment-2835753558

   > @CurtHagenlocher We should do 
[apache/arrow-dotnet#1](https://github.com/apache/arrow-dotnet/issues/1) after 
we merge this, right?
   
   Hi, have you got a chance to merge this code? I have another huge PR 
prepared? I'm waiting for 6 or more months for it.
   @jeremyosterhoudt can you assist please?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Re: [PR] GH-44800: [C#] Implement Flight SQL Client [arrow]

2025-04-10 Thread via GitHub 


zgramana commented on PR #44783:
URL: https://github.com/apache/arrow/pull/44783#issuecomment-2783805813

   > Thanks again for such a comprehensive change! I'm very sorry it took me 
this long to get to it. I've left a bunch of feedback of various degrees of 
seriousness. You'll also likely need to rebase given the amount of time that 
has passed (and the presence of a merge conflict).
   
   @CurtHagenlocher since this PR no longer has a merge conflict, and it get 
your final 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Re: [PR] GH-44800: [C#] Implement Flight SQL Client [arrow]

2025-04-07 Thread via GitHub 


kou commented on PR #44783:
URL: https://github.com/apache/arrow/pull/44783#issuecomment-2785066214

   @CurtHagenlocher We should do 
https://github.com/apache/arrow-dotnet/issues/1 after we merge this, right?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Re: [PR] GH-44800: [C#] Implement Flight SQL Client [arrow]

2025-04-07 Thread via GitHub 


zgramana commented on PR #44783:
URL: https://github.com/apache/arrow/pull/44783#issuecomment-2783802768

   > Thanks again for such a comprehensive change! I'm very sorry it took me 
this long to get to it. I've left a bunch of feedback of various degrees of 
seriousness. You'll also likely need to rebase given the amount of time that 
has passed (and the presence of a merge conflict).
   
   @CurtHagenlocher since this PR no longer has a merge conflict, and it get 
your final approval and merge? Much appreciated.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Re: [PR] GH-44800: [C#] Implement Flight SQL Client [arrow]

2025-03-03 Thread via GitHub 


CurtHagenlocher commented on code in PR #44783:
URL: https://github.com/apache/arrow/pull/44783#discussion_r1978568789


##
csharp/src/Apache.Arrow.Flight/FlightInfoCancelRequest.cs:
##
@@ -0,0 +1,38 @@
+using System;

Review Comment:
   The Apache header is still missing from these two files in the latest 
iteration. (I haven't yet had a chance to review any or the other changes in 
the latest iteration, but the missing copyright notice does produce a build 
failure.)



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Re: [PR] GH-44800: [C#] Implement Flight SQL Client [arrow]

2025-02-25 Thread via GitHub 


HackPoint commented on code in PR #44783:
URL: https://github.com/apache/arrow/pull/44783#discussion_r1969424656


##
csharp/test/Apache.Arrow.Flight.Sql.Tests/FlightSqlClientTests.cs:
##
@@ -0,0 +1,855 @@
+// Licensed to the Apache Software Foundation (ASF) under one or more
+// contributor license agreements. See the NOTICE file distributed with
+// this work for additional information regarding copyright ownership.
+// The ASF licenses this file to You under the Apache License, Version 2.0
+// (the "License"); you may not use this file except in compliance with
+// the License.  You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+using System;
+using System.Collections.Generic;
+using System.Linq;
+using System.Threading.Tasks;
+using Apache.Arrow.Flight.Client;
+using Apache.Arrow.Flight.Sql.Client;
+using Apache.Arrow.Flight.TestWeb;
+using Apache.Arrow.Types;
+using Arrow.Flight.Protocol.Sql;
+using Google.Protobuf;
+using Grpc.Core.Utils;
+using Xunit;
+
+namespace Apache.Arrow.Flight.Sql.Tests;
+
+public class FlightSqlClientTests : IDisposable
+{
+readonly TestFlightSqlWebFactory _testWebFactory;
+readonly FlightStore _flightStore;
+private readonly FlightSqlClient _flightSqlClient;
+private readonly FlightSqlTestUtils _testUtils;
+
+public FlightSqlClientTests()
+{
+_flightStore = new FlightStore();
+_testWebFactory = new TestFlightSqlWebFactory(_flightStore);
+FlightClient flightClient = new(_testWebFactory.GetChannel());
+_flightSqlClient = new FlightSqlClient(flightClient);
+
+_testUtils = new FlightSqlTestUtils(_testWebFactory, _flightStore);
+}
+
+#region Transactions
+
+[Fact]
+public async Task CommitTransactionAsync()
+{
+// Arrange
+string transactionId = "sample-transaction-id";
+var transaction = new Transaction(transactionId);
+
+// Act
+var streamCall = _flightSqlClient.CommitAsync(transaction);
+var result = await streamCall.ResponseStream.ToListAsync();
+
+// Assert
+Assert.NotNull(result);
+Assert.Equal(transaction.TransactionId, result.FirstOrDefault()?.Body);
+}
+
+[Fact]
+public async Task BeginTransactionAsync()
+{
+// Arrange
+string expectedTransactionId = "sample-transaction-id";
+
+// Act
+var transaction = await _flightSqlClient.BeginTransactionAsync();
+
+// Assert
+Assert.NotNull(transaction);
+Assert.Equal(ByteString.CopyFromUtf8(expectedTransactionId), 
transaction.TransactionId);
+}
+
+[Fact]
+public async Task RollbackTransactionAsync()
+{
+// Arrange
+string transactionId = "sample-transaction-id";
+var transaction = new Transaction(transactionId);
+
+// Act
+var streamCall = _flightSqlClient.RollbackAsync(transaction);
+var result = await streamCall.ResponseStream.ToListAsync();
+
+// Assert
+Assert.NotNull(transaction);
+Assert.Equal(result.FirstOrDefault()?.Body, transaction.TransactionId);
+}
+
+#endregion
+
+#region PreparedStatement
+
+[Fact]
+public async Task PreparedAsync()
+{
+// Arrange
+string query = "INSERT INTO users (id, name) VALUES (1, 'John Doe')";
+var transaction = new Transaction("sample-transaction-id");
+var flightDescriptor = 
FlightDescriptor.CreateCommandDescriptor("test");
+
+// Create a sample schema for the dataset and parameters
+var schema = new Schema.Builder()
+.Field(f => f.Name("id").DataType(Int32Type.Default))
+.Field(f => f.Name("name").DataType(StringType.Default))
+.Build();
+
+var recordBatch = new RecordBatch(schema, new Array[]
+{
+new Int32Array.Builder().Append(1).Build(),
+new StringArray.Builder().Append("John Doe").Build()
+}, 1);
+
+var flightHolder = new FlightHolder(flightDescriptor, schema, 
_testWebFactory.GetAddress());
+flightHolder.AddBatch(new RecordBatchWithMetadata(recordBatch));
+_flightStore.Flights.Add(flightDescriptor, flightHolder);
+
+var datasetSchemaBytes = SchemaExtensions.SerializeSchema(schema);
+var parameterSchemaBytes = SchemaExtensions.SerializeSchema(schema);
+
+var preparedStatementResponse = new ActionCreatePreparedStatementResult
+{
+PreparedStatementHandle = 
ByteString.CopyFromUtf8("prepared-handle"),
+DatasetSchema = ByteString.CopyF

Re: [PR] GH-44800: [C#] Implement Flight SQL Client [arrow]

2025-02-25 Thread via GitHub 


HackPoint commented on code in PR #44783:
URL: https://github.com/apache/arrow/pull/44783#discussion_r1969422951


##
csharp/src/Apache.Arrow.Flight.Sql/FlightExtensions.cs:
##
@@ -0,0 +1,81 @@
+// Licensed to the Apache Software Foundation (ASF) under one or more
+// contributor license agreements. See the NOTICE file distributed with
+// this work for additional information regarding copyright ownership.
+// The ASF licenses this file to You under the Apache License, Version 2.0
+// (the "License"); you may not use this file except in compliance with
+// the License.  You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+using System;
+using System.Collections.Generic;
+using Google.Protobuf;
+using Google.Protobuf.WellKnownTypes;
+
+namespace Apache.Arrow.Flight.Sql;
+
+internal static class FlightExtensions
+{
+public static byte[] PackAndSerialize(this IMessage command) => 
Any.Pack(command).ToByteArray();
+public static T ParseAndUnpack(this ByteString source) where T : 
IMessage, new() => Any.Parser.ParseFrom(source).Unpack();
+
+public static IEnumerable ExtractRowCount(this RecordBatch batch)

Review Comment:
   Added:
   Executes an update SQL command and returns the number of affected rows.
   
   But after looking at:
   ```c++
   ARROW_ASSIGN_OR_RAISE(auto updated_rows,
 sql_client->ExecuteUpdate({}, "UPDATE STATEMENT"));
   ```
   
   I created the same method and implemented it as is.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Re: [PR] GH-44800: [C#] Implement Flight SQL Client [arrow]

2025-02-25 Thread via GitHub 


HackPoint commented on code in PR #44783:
URL: https://github.com/apache/arrow/pull/44783#discussion_r1969398707


##
csharp/src/Apache.Arrow.Flight.Sql/FlightExtensions.cs:
##
@@ -0,0 +1,81 @@
+// Licensed to the Apache Software Foundation (ASF) under one or more
+// contributor license agreements. See the NOTICE file distributed with
+// this work for additional information regarding copyright ownership.
+// The ASF licenses this file to You under the Apache License, Version 2.0
+// (the "License"); you may not use this file except in compliance with
+// the License.  You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+using System;
+using System.Collections.Generic;
+using Google.Protobuf;
+using Google.Protobuf.WellKnownTypes;
+
+namespace Apache.Arrow.Flight.Sql;
+
+internal static class FlightExtensions
+{
+public static byte[] PackAndSerialize(this IMessage command) => 
Any.Pack(command).ToByteArray();
+public static T ParseAndUnpack(this ByteString source) where T : 
IMessage, new() => Any.Parser.ParseFrom(source).Unpack();
+
+public static IEnumerable ExtractRowCount(this RecordBatch batch)
+{
+foreach (var array in batch.Arrays)
+{
+var values = ExtractValues(array);
+foreach (var value in values)
+{
+yield return value switch
+{
+long l => l,
+int i => i != 0 ? i : 0,

Review Comment:
   Improved the implementation and removed at all this redundant implementation



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Re: [PR] GH-44800: [C#] Implement Flight SQL Client [arrow]

2025-02-24 Thread via GitHub 


HackPoint commented on code in PR #44783:
URL: https://github.com/apache/arrow/pull/44783#discussion_r1967861118


##
csharp/src/Apache.Arrow.Flight.Sql/Transaction.cs:
##
@@ -0,0 +1,48 @@
+// Licensed to the Apache Software Foundation (ASF) under one or more
+// contributor license agreements. See the NOTICE file distributed with
+// this work for additional information regarding copyright ownership.
+// The ASF licenses this file to You under the Apache License, Version 2.0
+// (the "License"); you may not use this file except in compliance with
+// the License.  You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+using Google.Protobuf;
+
+namespace Apache.Arrow.Flight.Sql;
+
+public class Transaction
+{
+private static readonly ByteString TransactionIdDefaultValue = 
ByteString.Empty;
+private ByteString? _transactionId;
+
+public ByteString TransactionId
+{
+get => _transactionId ?? TransactionIdDefaultValue;
+set => _transactionId = ProtoPreconditions.CheckNotNull(value, 
nameof(value));
+}
+
+public static readonly Transaction NoTransaction = 
new(TransactionIdDefaultValue);
+
+public Transaction(ByteString transactionId)
+{
+TransactionId = transactionId;
+}
+
+public Transaction(string transactionId)
+{
+_transactionId = ByteString.CopyFromUtf8(transactionId);
+}
+
+public bool IsValid() => TransactionId.Length > 0;
+public void ResetTransaction()

Review Comment:
   removed it from the implementation



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Re: [PR] GH-44800: [C#] Implement Flight SQL Client [arrow]

2025-02-24 Thread via GitHub 


HackPoint commented on code in PR #44783:
URL: https://github.com/apache/arrow/pull/44783#discussion_r1968080313


##
csharp/src/Apache.Arrow.Flight.Sql/FlightCallOptions.cs:
##
@@ -0,0 +1,47 @@
+// Licensed to the Apache Software Foundation (ASF) under one or more
+// contributor license agreements. See the NOTICE file distributed with
+// this work for additional information regarding copyright ownership.
+// The ASF licenses this file to You under the Apache License, Version 2.0
+// (the "License"); you may not use this file except in compliance with
+// the License.  You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+using System;
+using System.Buffers;
+using System.Threading;
+using Grpc.Core;
+
+namespace Apache.Arrow.Flight.Sql;
+
+public class FlightCallOptions
+{
+public FlightCallOptions()
+{
+Timeout = TimeSpan.FromSeconds(-1);
+}

Review Comment:
   done



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Re: [PR] GH-44800: [C#] Implement Flight SQL Client [arrow]

2025-02-24 Thread via GitHub 


HackPoint commented on code in PR #44783:
URL: https://github.com/apache/arrow/pull/44783#discussion_r1968081819


##
csharp/src/Apache.Arrow.Flight.Sql/FlightCallOptions.cs:
##
@@ -0,0 +1,47 @@
+// Licensed to the Apache Software Foundation (ASF) under one or more
+// contributor license agreements. See the NOTICE file distributed with
+// this work for additional information regarding copyright ownership.
+// The ASF licenses this file to You under the Apache License, Version 2.0
+// (the "License"); you may not use this file except in compliance with
+// the License.  You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+using System;
+using System.Buffers;
+using System.Threading;
+using Grpc.Core;
+
+namespace Apache.Arrow.Flight.Sql;
+
+public class FlightCallOptions
+{
+public FlightCallOptions()
+{
+Timeout = TimeSpan.FromSeconds(-1);
+}
+// Implement any necessary options for RPC calls
+public Metadata Headers { get; set; } = new();
+
+/// 
+/// Gets or sets a token to enable interactive user cancellation of 
long-running requests.
+/// 
+public CancellationToken StopToken { get; set; }

Review Comment:
   removed the property



##
csharp/src/Apache.Arrow.Flight.Sql/FlightCallOptions.cs:
##
@@ -0,0 +1,47 @@
+// Licensed to the Apache Software Foundation (ASF) under one or more
+// contributor license agreements. See the NOTICE file distributed with
+// this work for additional information regarding copyright ownership.
+// The ASF licenses this file to You under the Apache License, Version 2.0
+// (the "License"); you may not use this file except in compliance with
+// the License.  You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+using System;
+using System.Buffers;
+using System.Threading;
+using Grpc.Core;
+
+namespace Apache.Arrow.Flight.Sql;
+
+public class FlightCallOptions
+{
+public FlightCallOptions()
+{
+Timeout = TimeSpan.FromSeconds(-1);
+}
+// Implement any necessary options for RPC calls
+public Metadata Headers { get; set; } = new();
+
+/// 
+/// Gets or sets a token to enable interactive user cancellation of 
long-running requests.
+/// 
+public CancellationToken StopToken { get; set; }
+
+/// 
+/// Gets or sets the optional timeout for this call.
+/// Negative durations mean an implementation-defined default behavior 
will be used instead.
+/// 
+public TimeSpan Timeout { get; set; }
+
+/// 
+/// Gets or sets an optional memory manager to control where to allocate 
incoming data.
+/// 
+public MemoryManager? MemoryManager { get; set; }

Review Comment:
   removed the property



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Re: [PR] GH-44800: [C#] Implement Flight SQL Client [arrow]

2025-02-24 Thread via GitHub 


HackPoint commented on code in PR #44783:
URL: https://github.com/apache/arrow/pull/44783#discussion_r1968079460


##
csharp/src/Apache.Arrow.Flight.Sql/DoPutResult.cs:
##
@@ -0,0 +1,54 @@
+// Licensed to the Apache Software Foundation (ASF) under one or more
+// contributor license agreements. See the NOTICE file distributed with
+// this work for additional information regarding copyright ownership.
+// The ASF licenses this file to You under the Apache License, Version 2.0
+// (the "License"); you may not use this file except in compliance with
+// the License.  You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+using System.Threading.Tasks;
+using Apache.Arrow.Flight.Client;
+using Grpc.Core;
+
+namespace Apache.Arrow.Flight.Sql;
+
+public class DoPutResult
+{
+public FlightClientRecordBatchStreamWriter Writer { get; }
+public IAsyncStreamReader Reader { get; }
+
+public DoPutResult(FlightClientRecordBatchStreamWriter writer, 
IAsyncStreamReader reader)
+{
+Writer = writer;
+Reader = reader;
+}
+
+/// 
+/// Reads the metadata asynchronously from the reader.
+/// 
+/// A ByteString containing the metadata read from the 
reader.
+public async Task ReadMetadataAsync()

Review Comment:
   These are; Very short, CPU-bound calculations, thus is not required to put 
CancelationToken.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Re: [PR] GH-44800: [C#] Implement Flight SQL Client [arrow]

2025-02-24 Thread via GitHub 


HackPoint commented on code in PR #44783:
URL: https://github.com/apache/arrow/pull/44783#discussion_r1968073191


##
csharp/src/Apache.Arrow.Flight.Sql/PreparedStatement.cs:
##
@@ -0,0 +1,383 @@
+// Licensed to the Apache Software Foundation (ASF) under one or more
+// contributor license agreements. See the NOTICE file distributed with
+// this work for additional information regarding copyright ownership.
+// The ASF licenses this file to You under the Apache License, Version 2.0
+// (the "License"); you may not use this file except in compliance with
+// the License.  You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+using System;
+using System.Collections.Generic;
+using System.Linq;
+using System.Text;
+using System.Threading;
+using System.Threading.Tasks;
+using Apache.Arrow.Flight.Sql.Client;
+using Arrow.Flight.Protocol.Sql;
+using Google.Protobuf;
+using Grpc.Core;
+
+namespace Apache.Arrow.Flight.Sql;
+
+public class PreparedStatement : IDisposable
+{
+private readonly FlightSqlClient _client;
+private readonly string _handle;
+private Schema _datasetSchema;
+private Schema _parameterSchema;
+private RecordBatch? _recordsBatch;
+private bool _isClosed;
+public bool IsClosed => _isClosed;
+public string Handle => _handle;
+public RecordBatch? ParametersBatch => _recordsBatch;
+
+/// 
+/// Initializes a new instance of the  
class.
+/// 
+/// The Flight SQL client used for executing SQL 
operations.
+/// The handle representing the prepared 
statement.
+/// The schema of the result dataset.
+/// The schema of the parameters for this 
prepared statement.
+public PreparedStatement(FlightSqlClient client, string handle, Schema 
datasetSchema, Schema parameterSchema)
+{
+_client = client ?? throw new ArgumentNullException(nameof(client));
+_handle = handle ?? throw new ArgumentNullException(nameof(handle));
+_datasetSchema = datasetSchema ?? throw new 
ArgumentNullException(nameof(datasetSchema));
+_parameterSchema = parameterSchema ?? throw new 
ArgumentNullException(nameof(parameterSchema));
+_isClosed = false;
+}
+
+/// 
+/// Retrieves the schema associated with the prepared statement 
asynchronously.
+/// 
+/// The options used to configure the Flight 
call.
+/// A task representing the asynchronous operation, which returns 
the schema of the result set.
+/// Thrown when the schema is 
empty or invalid.
+public async Task GetSchemaAsync(FlightCallOptions? options = 
default)
+{
+EnsureStatementIsNotClosed();
+
+try
+{
+var command = new CommandPreparedStatementQuery
+{
+PreparedStatementHandle = ByteString.CopyFrom(_handle, 
Encoding.UTF8)
+};
+var descriptor = 
FlightDescriptor.CreateCommandDescriptor(command.PackAndSerialize());
+var schema = await _client.GetSchemaAsync(descriptor, 
options).ConfigureAwait(false);
+if (schema == null || !schema.FieldsList.Any())
+{
+throw new InvalidOperationException("Schema is empty or 
invalid.");
+}
+return schema;
+}
+catch (RpcException ex)
+{
+throw new InvalidOperationException("Failed to retrieve the schema 
for the prepared statement", ex);
+}
+}
+
+

Review Comment:
   don't the extra line, but removed one above.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Re: [PR] GH-44800: [C#] Implement Flight SQL Client [arrow]

2025-02-24 Thread via GitHub 


HackPoint commented on code in PR #44783:
URL: https://github.com/apache/arrow/pull/44783#discussion_r1968066965


##
csharp/src/Apache.Arrow.Flight.Sql/RecordBatchExtensions.cs:
##
@@ -0,0 +1,138 @@
+// Licensed to the Apache Software Foundation (ASF) under one or more
+// contributor license agreements. See the NOTICE file distributed with
+// this work for additional information regarding copyright ownership.
+// The ASF licenses this file to You under the Apache License, Version 2.0
+// (the "License"); you may not use this file except in compliance with
+// the License.  You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+using System;
+using System.Collections.Generic;
+using System.IO;
+using System.Linq;
+using System.Threading;
+using System.Threading.Channels;
+using System.Threading.Tasks;
+using Apache.Arrow.Ipc;
+using Google.Protobuf;
+using Grpc.Core;
+
+namespace Apache.Arrow.Flight.Sql;
+
+public static class RecordBatchExtensions
+{
+/// 
+/// Converts a RecordBatch into an asynchronous stream of FlightData.
+/// 
+/// The RecordBatch to convert.
+/// The FlightDescriptor describing the 
Flight data.
+/// An asynchronous stream of FlightData objects.
+public static async IAsyncEnumerable 
ToFlightDataStreamAsync(this RecordBatch recordBatch,
+FlightDescriptor flightDescriptor)
+{
+if (recordBatch == null)
+{
+throw new ArgumentNullException(nameof(recordBatch));
+}
+
+// Use a memory stream to write the Arrow RecordBatch into FlightData 
format
+using var memoryStream = new MemoryStream();
+var writer = new ArrowStreamWriter(memoryStream, recordBatch.Schema);
+
+// Write the RecordBatch to the stream
+await writer.WriteRecordBatchAsync(recordBatch).ConfigureAwait(false);
+await writer.WriteEndAsync().ConfigureAwait(false);
+
+// Reset the memory stream position
+memoryStream.Position = 0;
+
+// Read back the data to create FlightData
+var flightData = new FlightData(flightDescriptor, 
ByteString.CopyFrom(memoryStream.ToArray()),
+ByteString.CopyFrom(memoryStream.ToArray()));
+yield return flightData;
+}
+
+/// 
+/// Converts a RecordBatch into an IAsyncStreamReader.
+/// 
+/// The RecordBatch to convert.
+/// The FlightDescriptor describing the 
Flight data.
+/// An IAsyncStreamReader of FlightData.
+public static IAsyncStreamReader ToFlightDataStream(this 
RecordBatch recordBatch, FlightDescriptor flightDescriptor)
+{
+if (recordBatch == null) throw new 
ArgumentNullException(nameof(recordBatch));
+if (flightDescriptor == null) throw new 
ArgumentNullException(nameof(flightDescriptor));
+
+var channel = Channel.CreateUnbounded();
+
+try
+{
+if (recordBatch.Schema == null || 
!recordBatch.Schema.FieldsList.Any())
+{
+throw new InvalidOperationException("The record batch has an 
invalid or empty schema.");
+}
+
+using var memoryStream = new MemoryStream();
+using var writer = new ArrowStreamWriter(memoryStream, 
recordBatch.Schema);
+writer.WriteRecordBatch(recordBatch);
+writer.WriteEnd();
+memoryStream.Position = 0;
+var flightData = new FlightData(flightDescriptor, 
ByteString.CopyFrom(memoryStream.ToArray()), ByteString.Empty, 
ByteString.Empty);
+if (flightData.DataBody.IsEmpty)
+{
+throw new InvalidOperationException(
+"The generated FlightData is empty. Check the RecordBatch 
content.");
+}
+
+channel.Writer.TryWrite(flightData);

Review Comment:
   Implementation is deleted



##
csharp/src/Apache.Arrow.Flight.Sql/RecordBatchExtensions.cs:
##
@@ -0,0 +1,138 @@
+// Licensed to the Apache Software Foundation (ASF) under one or more
+// contributor license agreements. See the NOTICE file distributed with
+// this work for additional information regarding copyright ownership.
+// The ASF licenses this file to You under the Apache License, Version 2.0
+// (the "License"); you may not use this file except in compliance with
+// the License.  You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either

Re: [PR] GH-44800: [C#] Implement Flight SQL Client [arrow]

2025-02-24 Thread via GitHub 


HackPoint commented on code in PR #44783:
URL: https://github.com/apache/arrow/pull/44783#discussion_r1968063719


##
csharp/src/Apache.Arrow.Flight.Sql/RecordBatchExtensions.cs:
##
@@ -0,0 +1,138 @@
+// Licensed to the Apache Software Foundation (ASF) under one or more
+// contributor license agreements. See the NOTICE file distributed with
+// this work for additional information regarding copyright ownership.
+// The ASF licenses this file to You under the Apache License, Version 2.0
+// (the "License"); you may not use this file except in compliance with
+// the License.  You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+using System;
+using System.Collections.Generic;
+using System.IO;
+using System.Linq;
+using System.Threading;
+using System.Threading.Channels;
+using System.Threading.Tasks;
+using Apache.Arrow.Ipc;
+using Google.Protobuf;
+using Grpc.Core;
+
+namespace Apache.Arrow.Flight.Sql;
+
+public static class RecordBatchExtensions
+{
+/// 
+/// Converts a RecordBatch into an asynchronous stream of FlightData.
+/// 
+/// The RecordBatch to convert.
+/// The FlightDescriptor describing the 
Flight data.
+/// An asynchronous stream of FlightData objects.
+public static async IAsyncEnumerable 
ToFlightDataStreamAsync(this RecordBatch recordBatch,
+FlightDescriptor flightDescriptor)
+{
+if (recordBatch == null)
+{
+throw new ArgumentNullException(nameof(recordBatch));
+}
+
+// Use a memory stream to write the Arrow RecordBatch into FlightData 
format
+using var memoryStream = new MemoryStream();
+var writer = new ArrowStreamWriter(memoryStream, recordBatch.Schema);
+
+// Write the RecordBatch to the stream
+await writer.WriteRecordBatchAsync(recordBatch).ConfigureAwait(false);
+await writer.WriteEndAsync().ConfigureAwait(false);
+
+// Reset the memory stream position
+memoryStream.Position = 0;
+
+// Read back the data to create FlightData
+var flightData = new FlightData(flightDescriptor, 
ByteString.CopyFrom(memoryStream.ToArray()),

Review Comment:
   I fully removed this class from implementation.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Re: [PR] GH-44800: [C#] Implement Flight SQL Client [arrow]

2025-02-24 Thread via GitHub 


HackPoint commented on code in PR #44783:
URL: https://github.com/apache/arrow/pull/44783#discussion_r1968061250


##
csharp/src/Apache.Arrow.Flight.Sql/SchemaExtensions.cs:
##
@@ -0,0 +1,52 @@
+// Licensed to the Apache Software Foundation (ASF) under one or more
+// contributor license agreements. See the NOTICE file distributed with
+// this work for additional information regarding copyright ownership.
+// The ASF licenses this file to You under the Apache License, Version 2.0
+// (the "License"); you may not use this file except in compliance with
+// the License.  You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+using System;
+using System.IO;
+using Apache.Arrow.Ipc;
+
+namespace Apache.Arrow.Flight.Sql;
+
+public static class SchemaExtensions
+{
+/// 
+/// Deserializes a schema from a byte array.
+/// 
+/// The byte array representing the 
serialized schema.
+/// The deserialized Schema object.
+public static Schema DeserializeSchema(byte[] serializedSchema)
+{
+if (serializedSchema == null || serializedSchema.Length == 0)
+{
+throw new ArgumentException("Invalid serialized schema");
+}
+
+using var stream = new MemoryStream(serializedSchema);
+var reader = new ArrowStreamReader(stream);

Review Comment:
   Done



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Re: [PR] GH-44800: [C#] Implement Flight SQL Client [arrow]

2025-02-24 Thread via GitHub 


HackPoint commented on code in PR #44783:
URL: https://github.com/apache/arrow/pull/44783#discussion_r1968064767


##
csharp/src/Apache.Arrow.Flight.Sql/RecordBatchExtensions.cs:
##
@@ -0,0 +1,138 @@
+// Licensed to the Apache Software Foundation (ASF) under one or more
+// contributor license agreements. See the NOTICE file distributed with
+// this work for additional information regarding copyright ownership.
+// The ASF licenses this file to You under the Apache License, Version 2.0
+// (the "License"); you may not use this file except in compliance with
+// the License.  You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+using System;
+using System.Collections.Generic;
+using System.IO;
+using System.Linq;
+using System.Threading;
+using System.Threading.Channels;
+using System.Threading.Tasks;
+using Apache.Arrow.Ipc;
+using Google.Protobuf;
+using Grpc.Core;
+
+namespace Apache.Arrow.Flight.Sql;
+
+public static class RecordBatchExtensions
+{
+/// 
+/// Converts a RecordBatch into an asynchronous stream of FlightData.
+/// 
+/// The RecordBatch to convert.
+/// The FlightDescriptor describing the 
Flight data.
+/// An asynchronous stream of FlightData objects.
+public static async IAsyncEnumerable 
ToFlightDataStreamAsync(this RecordBatch recordBatch,
+FlightDescriptor flightDescriptor)
+{
+if (recordBatch == null)
+{
+throw new ArgumentNullException(nameof(recordBatch));
+}
+
+// Use a memory stream to write the Arrow RecordBatch into FlightData 
format
+using var memoryStream = new MemoryStream();
+var writer = new ArrowStreamWriter(memoryStream, recordBatch.Schema);
+
+// Write the RecordBatch to the stream
+await writer.WriteRecordBatchAsync(recordBatch).ConfigureAwait(false);
+await writer.WriteEndAsync().ConfigureAwait(false);
+
+// Reset the memory stream position
+memoryStream.Position = 0;
+
+// Read back the data to create FlightData
+var flightData = new FlightData(flightDescriptor, 
ByteString.CopyFrom(memoryStream.ToArray()),
+ByteString.CopyFrom(memoryStream.ToArray()));

Review Comment:
   Implementation is deleted 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Re: [PR] GH-44800: [C#] Implement Flight SQL Client [arrow]

2025-02-24 Thread via GitHub 


HackPoint commented on code in PR #44783:
URL: https://github.com/apache/arrow/pull/44783#discussion_r1968049070


##
csharp/src/Apache.Arrow.Flight.Sql/Transaction.cs:
##
@@ -0,0 +1,48 @@
+// Licensed to the Apache Software Foundation (ASF) under one or more
+// contributor license agreements. See the NOTICE file distributed with
+// this work for additional information regarding copyright ownership.
+// The ASF licenses this file to You under the Apache License, Version 2.0
+// (the "License"); you may not use this file except in compliance with
+// the License.  You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+using Google.Protobuf;
+
+namespace Apache.Arrow.Flight.Sql;
+
+public class Transaction

Review Comment:
   After re-evaluation, I've changed the Transaction to be value type for the 
moment be, 
   Seems that there's no too much data ( properties ) are being passed.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Re: [PR] GH-44800: [C#] Implement Flight SQL Client [arrow]

2025-02-08 Thread via GitHub 


CurtHagenlocher commented on code in PR #44783:
URL: https://github.com/apache/arrow/pull/44783#discussion_r1947995089


##
csharp/src/Apache.Arrow.Flight.Sql/TableRef.cs:
##
@@ -0,0 +1,23 @@
+// Licensed to the Apache Software Foundation (ASF) under one or more
+// contributor license agreements. See the NOTICE file distributed with
+// this work for additional information regarding copyright ownership.
+// The ASF licenses this file to You under the Apache License, Version 2.0
+// (the "License"); you may not use this file except in compliance with
+// the License.  You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+namespace Apache.Arrow.Flight.Sql;
+
+public class TableRef
+{
+public string? Catalog { get; set; }
+public string DbSchema { get; set; } = null!;

Review Comment:
   The `null!` seems dodgy. As these are required to be set, consider using a 
pair of constructors `TableRef(string, string)` and `TableRef(string, string, 
string)` instead.



##
csharp/src/Apache.Arrow.Flight.Sql/SchemaExtensions.cs:
##
@@ -0,0 +1,52 @@
+// Licensed to the Apache Software Foundation (ASF) under one or more
+// contributor license agreements. See the NOTICE file distributed with
+// this work for additional information regarding copyright ownership.
+// The ASF licenses this file to You under the Apache License, Version 2.0
+// (the "License"); you may not use this file except in compliance with
+// the License.  You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+using System;
+using System.IO;
+using Apache.Arrow.Ipc;
+
+namespace Apache.Arrow.Flight.Sql;
+
+public static class SchemaExtensions
+{
+/// 
+/// Deserializes a schema from a byte array.
+/// 
+/// The byte array representing the 
serialized schema.
+/// The deserialized Schema object.
+public static Schema DeserializeSchema(byte[] serializedSchema)
+{
+if (serializedSchema == null || serializedSchema.Length == 0)
+{
+throw new ArgumentException("Invalid serialized schema");
+}
+
+using var stream = new MemoryStream(serializedSchema);
+var reader = new ArrowStreamReader(stream);

Review Comment:
   There's a ctor for `ArrowStreamReader` which takes a `ReadOnlyMemory` 
as an argument, so you can just pass in the bytes. This is probably a bit more 
efficient.



##
csharp/src/Apache.Arrow.Flight.Sql/DoPutResult.cs:
##
@@ -0,0 +1,54 @@
+// Licensed to the Apache Software Foundation (ASF) under one or more
+// contributor license agreements. See the NOTICE file distributed with
+// this work for additional information regarding copyright ownership.
+// The ASF licenses this file to You under the Apache License, Version 2.0
+// (the "License"); you may not use this file except in compliance with
+// the License.  You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+using System.Threading.Tasks;
+using Apache.Arrow.Flight.Client;
+using Grpc.Core;
+
+namespace Apache.Arrow.Flight.Sql;
+
+public class DoPutResult
+{
+public FlightClientRecordBatchStreamWriter Writer { get; }
+public IAsyncStreamReader Reader { get; }
+
+public DoPutResult(FlightClientRecordBatchStreamWriter writer, 
IAsyncStreamReader reader)
+{
+Writer = writer;
+Reader = reader;
+}
+
+/// 
+/// Reads the metadata asynchronously from the reader.
+/// 
+/// A ByteString containing the metadata read from the 
reader.
+public async Task ReadMetadataAsync()

Review Comment:
   In general, none of the `async` methods in this change take a 
`CancellationToken`. Shouldn't they?



##
csharp/test/Apache.Arrow.Flight.Sql.Tests/FlightSqlClientTests.cs:
##
@@ -0,0 +1,855 @@
+// Licensed to the Apache Software Foundation (ASF) under one or more
+// contributor license agreements. See

Re: [PR] GH-44800: [C#] Implement Flight SQL Client [arrow]

2025-01-27 Thread via GitHub 


HackPoint commented on PR #44783:
URL: https://github.com/apache/arrow/pull/44783#issuecomment-2616200400

   > > Any updates on this? We are also looking to use .NET with a Flight SQL 
client and are eager to get this functionality.
   > 
   > I'm sorry about the delay; I was very much wanting to get the review done 
before today's fork but first work, then vacation, then a lengthy internet 
outage at home got in the way. I will definitely make time for a review later 
this week.
   
   Hi Curt, 
   Is it possible to review and if approved merge my PR, it's already waiting 
for more than 2 months.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Re: [PR] GH-44800: [C#] Implement Flight SQL Client [arrow]

2025-01-06 Thread via GitHub 


CurtHagenlocher commented on PR #44783:
URL: https://github.com/apache/arrow/pull/44783#issuecomment-2573692881

   > Any updates on this? We are also looking to use .NET with a Flight SQL 
client and are eager to get this functionality.
   
   I'm sorry about the delay; I was very much wanting to get the review done 
before today's fork but first work, then vacation, then a lengthy internet 
outage at home got in the way. I will definitely make time for a review later 
this week.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Re: [PR] GH-44800: [C#] Implement Flight SQL Client [arrow]

2025-01-06 Thread via GitHub 


jeremyosterhoudt commented on PR #44783:
URL: https://github.com/apache/arrow/pull/44783#issuecomment-2573618767

   Any updates on this?  We are also looking to use .NET with a Flight SQL 
client and are eager to get this functionality.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Re: [PR] GH-44800: [C#] Implement Flight SQL Client [arrow]

2024-11-28 Thread via GitHub 


HackPoint commented on PR #44783:
URL: https://github.com/apache/arrow/pull/44783#issuecomment-2505806254

   > Could you revert all `cpp/` changes?
   
   seems that it's done


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Re: [PR] GH-44800: [C#] Implement Flight SQL Client [arrow]

2024-11-25 Thread via GitHub 


kou commented on PR #44783:
URL: https://github.com/apache/arrow/pull/44783#issuecomment-2499692202

   Could you revert all `cpp/` changes?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Re: [PR] GH-44800: [C#] Implement Flight SQL Client [arrow]

2024-11-25 Thread via GitHub 


HackPoint commented on PR #44783:
URL: https://github.com/apache/arrow/pull/44783#issuecomment-2499695614

   I’ve haven’t made any changes in c++, but I’ll re-fork and cherry-pick my
   code into a new fork and push all the changes.
   
   On Tue, 26 Nov 2024 at 7:28 Sutou Kouhei ***@***.***> wrote:
   
   > Could you revert all cpp/ changes?
   >
   > —
   > Reply to this email directly, view it on GitHub
   > , or
   > unsubscribe
   > 

   > .
   > You are receiving this because you were mentioned.Message ID:
   > ***@***.***>
   >
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Re: [PR] GH-44800: [C#] Implement Flight SQL Client [arrow]

2024-11-25 Thread via GitHub 


CurtHagenlocher commented on PR #44783:
URL: https://github.com/apache/arrow/pull/44783#issuecomment-2498446618

   > Can you please review the code and allow me to close the PR please?
   
   I will review this tonight.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Re: [PR] GH-44800: [C#] Implement Flight SQL Client [arrow]

2024-11-25 Thread via GitHub 


HackPoint commented on PR #44783:
URL: https://github.com/apache/arrow/pull/44783#issuecomment-2498316865

   > I approved the workflows.
   > 
   > I don't work on C# sharp but wanted to thank you @HackPoint for such a big 
first-time contribution!
   
   Can you please review the code and allow me to close the PR please?
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Re: [PR] GH-44800: [C#] Implement Flight SQL Client [arrow]

2024-11-20 Thread via GitHub 


assignUser commented on PR #44783:
URL: https://github.com/apache/arrow/pull/44783#issuecomment-2489218881

   I approved the workflows.
   
   I don't work on C# sharp but wanted to thank you @HackPoint for such a big 
first-time contribution! 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Re: [PR] GH-44800: [C#] Implement Flight SQL Client [arrow]

2024-11-20 Thread via GitHub 


github-actions[bot] commented on PR #44783:
URL: https://github.com/apache/arrow/pull/44783#issuecomment-2489188280

   :warning: GitHub issue #44800 **has been automatically assigned in GitHub** 
to PR creator.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]