westonpace commented on code in PR #36550:
URL: https://github.com/apache/arrow/pull/36550#discussion_r1256029513
##########
python/pyarrow/_dataset.pyx:
##########
@@ -2988,7 +3017,7 @@ cdef class FileSystemFactoryOptions(_Weakrefable):
c_factory = self.options.partitioning.factory()
if c_factory.get() == nullptr:
return None
- return PartitioningFactory.wrap(c_factory)
+ return PartitioningFactory.wrap(c_factory, None, None)
Review Comment:
Won't this mean that this object can't be pickled?
##########
python/pyarrow/_dataset.pyx:
##########
@@ -2571,7 +2581,15 @@ cdef class DirectoryPartitioning(KeyValuePartitioning):
c_options.segment_encoding = _get_segment_encoding(segment_encoding)
return PartitioningFactory.wrap(
- CDirectoryPartitioning.MakeFactory(c_field_names, c_options))
+ CDirectoryPartitioning.MakeFactory(c_field_names, c_options),
+ _constructor_directory_partitioning_factory,
+ (field_names, infer_dictionary, max_partition_dictionary_size,
+ schema, segment_encoding)
+ )
+
+
+def _constructor_hive_partitioning_factory(*args):
+ return HivePartitioning.discover(*args)
Review Comment:
Does this method actually run discovery or does it just create the factory?
##########
python/pyarrow/_dataset_parquet.pyx:
##########
@@ -805,7 +805,7 @@ cdef class ParquetFactoryOptions(_Weakrefable):
c_factory = self.options.partitioning.factory()
if c_factory.get() == nullptr:
return None
- return PartitioningFactory.wrap(c_factory)
+ return PartitioningFactory.wrap(c_factory, None, None)
Review Comment:
Same question as above.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
