orhankislal commented on a change in pull request #439: DL: Add support in preprocessor to evenly distribute data for GPDB URL: https://github.com/apache/madlib/pull/439#discussion_r322392420
########## File path: src/ports/postgres/modules/deep_learning/test/input_data_preprocessor.sql_in ########## @@ -88,6 +89,23 @@ SELECT training_preprocessor_dl( 'label', 'x'); +-- Test data is evenly distributed across all segments (GPDB only) +m4_changequote(`<!', `!>') +m4_ifdef(<!__POSTGRESQL__!>, <!!>, <! +DROP TABLE IF EXISTS data_preprocessor_input_batch, data_preprocessor_input_batch_summary; +SELECT training_preprocessor_dl( + 'data_preprocessor_input', + 'data_preprocessor_input_batch', + 'id', + 'x', + 1); + +SELECT assert(count(*)=(SELECT ceil(17.0/count(*)) from gp_segment_configuration WHERE role = 'p' and content != -1), 'Even distribution of buffers failed.') +FROM data_preprocessor_input_batch +WHERE gp_segment_id = 0 +GROUP BY gp_segment_id; Review comment: If we have the `WHERE gp_segment_id = 0` clause, why do we need the `group by`? ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
