fmcquillan99 commented on issue #16: MADlib image loader: load dataset from disk URL: https://github.com/apache/madlib-site/pull/16#issuecomment-513045886 This might be OK, but wondering why (1) if I load from filesystem: ``` %sql drop table if exists cifar_10_train_data_filesystem: # Load images from file system iloader.load_dataset_from_disk('/Users/fmcquillan/tmp/cifar10', 'cifar_10_train_datafile_system', num_labels='all', append=False, no_temp_files=False) ``` I get this format: ``` cifar_10_train_data filesystem {{{{218,239,244},{219,240,245},{220,241,246}, ... ``` (2) and if I load from numpy: ``` # Load dataset into np array (x_train, y_train), (x_test, y_test) = cifar10.load_data() %sql DROP TABLE IF EXISTS cifar_10_train_data, cifar_10_test_data; # Save images to temporary directories and load into database iloader.load_dataset_from_np(x_train, y_train, 'cifar_10_train_data', append=False, no_temp_files=False) iloader.load_dataset_from_np(x_test, y_test, 'cifar_10_test_data', append=False, no_temp_files=False) ``` I get this format: ``` cifar_10_train_data {{{59,62,63},{43,46,45},{50,48,43}, ... ``` It means if I run the model architecture from https://keras.io/examples/cifar10_cnn/ it works when loading from numpy (2) but when I run from on filesystem loaded data (1) I get this message: ``` ERROR: plpy.Error: model_keras error: Input shape [32, 32, 3] in the model architecture does not match the input shape [1, 32, 32] of column independent_var in table cifar_10_train_data_packed. CONTEXT: Traceback (most recent call last): PL/Python function "madlib_keras_fit", line 21, in <module> madlib_keras.fit(**globals()) PL/Python function "madlib_keras_fit", line 42, in wrapper PL/Python function "madlib_keras_fit", line 102, in fit PL/Python function "madlib_keras_fit", line 308, in validate_input_shapes PL/Python function "madlib_keras_fit", line 86, in _validate_input_shapes PL/Python function "madlib_keras_fit" ```
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
