marcoabreu commented on a change in pull request #16175: [Dataset] add shard API
URL: https://github.com/apache/incubator-mxnet/pull/16175#discussion_r324882558
##########
File path: tests/python/unittest/test_gluon_data.py
##########
@@ -296,6 +296,24 @@ def test_dataset_filter():
for idx, sample in enumerate(a_xform_filtered):
assert sample % 10 == 0
+def test_dataset_shard():
+ length = 9
+ a = mx.gluon.data.SimpleDataset([i for i in range(length)])
+ shard_0 = a.shard(4, 0)
+ shard_1 = a.shard(4, 1)
+ shard_2 = a.shard(4, 2)
+ shard_3 = a.shard(4, 3)
+ assert len(shard_0) + len(shard_1) + len(shard_2) + len(shard_3) == length
+ assert len(shard_0) > 0
Review comment:
Use == instead since the size should be known?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
