jorgecarleitao commented on a change in pull request #8222:
URL: https://github.com/apache/arrow/pull/8222#discussion_r501440700
##########
File path: rust/datafusion/src/test/mod.rs
##########
@@ -135,6 +135,13 @@ pub fn format_batch(batch: &RecordBatch) -> Vec<String> {
}
let array = batch.column(column_index);
match array.data_type() {
+ DataType::Utf8 => s.push_str(
+ array
+ .as_any()
+ .downcast_ref::<array::StringArray>()
+ .unwrap()
+ .value(row_index),
+ ),
Review comment:
I am really sorry, I misread this. What I meant was not to test the
`format_batch` function, but to test `count distinct` in integration.
However, I see that this was not the module I though it was. The code in
this module is perfect as is now.
I meant adding a test
[here](https://github.com/apache/arrow/blob/master/rust/datafusion/tests/sql.rs),
which is where we perform integration tests.
Something along the lines of:
```rust
#[tokio::test]
async fn query_count_distinct() -> Result<()> {
let schema = Arc::new(Schema::new(vec![
Field::new("c1", DataType::Int32, true),
]));
let data = RecordBatch::try_new(
schema.clone(),
vec![
Arc::new(Int32Array::from(vec![Some(0), Some(1), None, Some(3),
Some(3)])),
],
)?;
let table = MemTable::new(schema, vec![vec![data]])?;
let mut ctx = ExecutionContext::new();
ctx.register_table("test", Box::new(table));
let sql = "SELECT COUNT(DISTINCT c1) FROM test";
let actual = execute(&mut ctx, sql).await;
let expected = vec![
vec!["0", "1"],
vec!["1", "1"],
vec!["3", "2"],
];
assert_eq!(expected, actual);
Ok(())
}
```
does this make sense?
##########
File path: rust/datafusion/src/test/mod.rs
##########
@@ -135,6 +135,13 @@ pub fn format_batch(batch: &RecordBatch) -> Vec<String> {
}
let array = batch.column(column_index);
match array.data_type() {
+ DataType::Utf8 => s.push_str(
+ array
+ .as_any()
+ .downcast_ref::<array::StringArray>()
+ .unwrap()
+ .value(row_index),
+ ),
Review comment:
I think that `count(distinct)` should be null when all entries are null,
for consistency with `count`. I do not think that this blocks this, though. It
is an edge case IMO.
##########
File path: rust/datafusion/src/test/mod.rs
##########
@@ -135,6 +135,13 @@ pub fn format_batch(batch: &RecordBatch) -> Vec<String> {
}
let array = batch.column(column_index);
match array.data_type() {
+ DataType::Utf8 => s.push_str(
+ array
+ .as_any()
+ .downcast_ref::<array::StringArray>()
+ .unwrap()
+ .value(row_index),
+ ),
Review comment:
I think that `count(distinct)` should be null when all entries are null,
for consistency with `count`. I do not think that it blocks this, though. It is
an edge case IMO.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
