[GitHub] [tvm] leowang1225 commented on a change in pull request #7185: [AutoScheduler] Add custom build function
leowang1225 commented on a change in pull request #7185: URL: https://github.com/apache/tvm/pull/7185#discussion_r551682259 ## File path: python/tvm/auto_scheduler/measure.py ## @@ -303,12 +315,28 @@ class LocalBuilder(ProgramBuilder): This is used in a wrapper of the multiprocessing.Process.join(). n_parallel : int = multiprocessing.cpu_count() Number of threads used to build in parallel. -build_func : str = 'default' -The name of registered build function. +build_func: callable or str = "default" +If is 'default', use default build function +If is 'ndk', use function for android ndk +If is callable, use it as custom build function, expect lib_format field. """ def __init__(self, timeout=15, n_parallel=multiprocessing.cpu_count(), build_func="default"): -self.__init_handle_by_constructor__(_ffi_api.LocalBuilder, timeout, n_parallel, build_func) +if build_func == "default": +BuildFunc.name = "default" +BuildFunc.build_func = tar.tar +elif build_func == "ndk": +BuildFunc.name = "ndk" +BuildFunc.build_func = ndk.create_shared +elif not isinstance(build_func, str): +BuildFunc.name = "custom" +BuildFunc.build_func = build_func Review comment: OK, I have already modified This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [tvm] leowang1225 commented on a change in pull request #7185: [AutoScheduler] Add custom build function
leowang1225 commented on a change in pull request #7185: URL: https://github.com/apache/tvm/pull/7185#discussion_r551207425 ## File path: python/tvm/auto_scheduler/measure.py ## @@ -624,12 +652,10 @@ def local_build_worker(args): The build result of this Builder thread. """ inp, build_func, timeout, verbose = args -if build_func == "default": -build_func = tar.tar -elif build_func == "ndk": -build_func = ndk.create_shared -else: -raise ValueError("Invalid build_func" + build_func) +assert build_func == BuildFunc.name, ( +"BuildFunc.name: " + BuildFunc.name + ", but args is: " + build_func +) +build_func = BuildFunc.build_func Review comment: ` def cross_compiler( compile_func, options=None, output_format=None, get_target_triple=None, add_files=None ): def _fcompile(outputs, objects, options=None): all_options = base_options if options is not None: all_options += options compile_func(outputs, objects + add_files, options=all_options, **kwargs) if not output_format and hasattr(compile_func, "output_format"): output_format = compile_func.output_format output_format = output_format if output_format else "so" if not get_target_triple and hasattr(compile_func, "get_target_triple"): get_target_triple = compile_func.get_target_triple _fcompile.output_format = output_format _fcompile.get_target_triple = get_target_triple return _fcompile ` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [tvm] leowang1225 commented on a change in pull request #7185: [AutoScheduler] Add custom build function
leowang1225 commented on a change in pull request #7185: URL: https://github.com/apache/tvm/pull/7185#discussion_r551207425 ## File path: python/tvm/auto_scheduler/measure.py ## @@ -624,12 +652,10 @@ def local_build_worker(args): The build result of this Builder thread. """ inp, build_func, timeout, verbose = args -if build_func == "default": -build_func = tar.tar -elif build_func == "ndk": -build_func = ndk.create_shared -else: -raise ValueError("Invalid build_func" + build_func) +assert build_func == BuildFunc.name, ( +"BuildFunc.name: " + BuildFunc.name + ", but args is: " + build_func +) +build_func = BuildFunc.build_func Review comment: ` def cross_compiler( compile_func, options=None, output_format=None, get_target_triple=None, add_files=None ): def _fcompile(outputs, objects, options=None): all_options = base_options if options is not None: all_options += options compile_func(outputs, objects + add_files, options=all_options, **kwargs) if not output_format and hasattr(compile_func, "output_format"): output_format = compile_func.output_format output_format = output_format if output_format else "so" if not get_target_triple and hasattr(compile_func, "get_target_triple"): get_target_triple = compile_func.get_target_triple _fcompile.output_format = output_format _fcompile.get_target_triple = get_target_triple return _fcompile ` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [tvm] leowang1225 commented on a change in pull request #7185: [AutoScheduler] Add custom build function
leowang1225 commented on a change in pull request #7185: URL: https://github.com/apache/tvm/pull/7185#discussion_r551207425 ## File path: python/tvm/auto_scheduler/measure.py ## @@ -624,12 +652,10 @@ def local_build_worker(args): The build result of this Builder thread. """ inp, build_func, timeout, verbose = args -if build_func == "default": -build_func = tar.tar -elif build_func == "ndk": -build_func = ndk.create_shared -else: -raise ValueError("Invalid build_func" + build_func) +assert build_func == BuildFunc.name, ( +"BuildFunc.name: " + BuildFunc.name + ", but args is: " + build_func +) +build_func = BuildFunc.build_func Review comment: ` def cross_compiler( compile_func, options=None, output_format=None, get_target_triple=None, add_files=None ): ... def _fcompile(outputs, objects, options=None): all_options = base_options if options is not None: all_options += options compile_func(outputs, objects + add_files, options=all_options, **kwargs) if not output_format and hasattr(compile_func, "output_format"): output_format = compile_func.output_format output_format = output_format if output_format else "so" if not get_target_triple and hasattr(compile_func, "get_target_triple"): get_target_triple = compile_func.get_target_triple _fcompile.output_format = output_format _fcompile.get_target_triple = get_target_triple return _fcompile ` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [tvm] leowang1225 commented on a change in pull request #7185: [AutoScheduler] Add custom build function
leowang1225 commented on a change in pull request #7185: URL: https://github.com/apache/tvm/pull/7185#discussion_r551195931 ## File path: python/tvm/auto_scheduler/measure.py ## @@ -624,12 +652,10 @@ def local_build_worker(args): The build result of this Builder thread. """ inp, build_func, timeout, verbose = args -if build_func == "default": -build_func = tar.tar -elif build_func == "ndk": -build_func = ndk.create_shared -else: -raise ValueError("Invalid build_func" + build_func) +assert build_func == BuildFunc.name, ( +"BuildFunc.name: " + BuildFunc.name + ", but args is: " + build_func +) +build_func = BuildFunc.build_func Review comment: yes This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [tvm] leowang1225 commented on a change in pull request #7185: [AutoScheduler] Add custom build function
leowang1225 commented on a change in pull request #7185: URL: https://github.com/apache/tvm/pull/7185#discussion_r551170406 ## File path: python/tvm/auto_scheduler/measure.py ## @@ -624,12 +652,10 @@ def local_build_worker(args): The build result of this Builder thread. """ inp, build_func, timeout, verbose = args -if build_func == "default": -build_func = tar.tar -elif build_func == "ndk": -build_func = ndk.create_shared -else: -raise ValueError("Invalid build_func" + build_func) +assert build_func == BuildFunc.name, ( +"BuildFunc.name: " + BuildFunc.name + ", but args is: " + build_func +) +build_func = BuildFunc.build_func Review comment: @jcf94 I try use @tvm._ffi.register_func, but the custom build func is a python callable, it is not just function name. we need the whole python callable context, and the callable also have attribute. like cross_compiler, it has output_format and get_target_triple attributes. _ use_ndk = True build_func = cc.cross_compiler(ndk.create_shared, options=ndk_options) tuner = auto_scheduler.TaskScheduler(tasks, task_weights) tune_option = auto_scheduler.TuningOptions( num_measure_trials=200, # change this to 2 to achieve the best performance builder=auto_scheduler.LocalBuilder(build_func=build_func if use_ndk else "default"), runner=auto_scheduler.RPCRunner( device_key, host=tracker_ip, port=tracker_port, repeat=3, timeout=50 ), measure_callbacks=[auto_scheduler.RecordToFile(log_file)], ) _ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [tvm] leowang1225 commented on a change in pull request #7185: [AutoScheduler] Add custom build function
leowang1225 commented on a change in pull request #7185: URL: https://github.com/apache/tvm/pull/7185#discussion_r551170406 ## File path: python/tvm/auto_scheduler/measure.py ## @@ -624,12 +652,10 @@ def local_build_worker(args): The build result of this Builder thread. """ inp, build_func, timeout, verbose = args -if build_func == "default": -build_func = tar.tar -elif build_func == "ndk": -build_func = ndk.create_shared -else: -raise ValueError("Invalid build_func" + build_func) +assert build_func == BuildFunc.name, ( +"BuildFunc.name: " + BuildFunc.name + ", but args is: " + build_func +) +build_func = BuildFunc.build_func Review comment: @jcf94 I try use @tvm._ffi.register_func, but the custom build func is a python callable, it is not just function name. we need the whole python callable context, and the callable also have attribute. like cross_compiler, it has output_format and get_target_triple attributes. use_ndk = True build_func = cc.cross_compiler(ndk.create_shared, options=ndk_options) tuner = auto_scheduler.TaskScheduler(tasks, task_weights) tune_option = auto_scheduler.TuningOptions( num_measure_trials=200, # change this to 2 to achieve the best performance builder=auto_scheduler.LocalBuilder(build_func=build_func if use_ndk else "default"), runner=auto_scheduler.RPCRunner( device_key, host=tracker_ip, port=tracker_port, repeat=3, timeout=50 ), measure_callbacks=[auto_scheduler.RecordToFile(log_file)], ) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [tvm] leowang1225 commented on a change in pull request #7185: [AutoScheduler] Add custom build function
leowang1225 commented on a change in pull request #7185: URL: https://github.com/apache/tvm/pull/7185#discussion_r551170406 ## File path: python/tvm/auto_scheduler/measure.py ## @@ -624,12 +652,10 @@ def local_build_worker(args): The build result of this Builder thread. """ inp, build_func, timeout, verbose = args -if build_func == "default": -build_func = tar.tar -elif build_func == "ndk": -build_func = ndk.create_shared -else: -raise ValueError("Invalid build_func" + build_func) +assert build_func == BuildFunc.name, ( +"BuildFunc.name: " + BuildFunc.name + ", but args is: " + build_func +) +build_func = BuildFunc.build_func Review comment: @jcf94 I try use @tvm._ffi.register_func, but the custom build func is a python callable, it is not just function. we need the whole python callable context, build_func = cc.cross_compiler(ndk.create_shared, options=ndk_options) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [tvm] leowang1225 commented on a change in pull request #7185: [AutoScheduler] Add custom build function
leowang1225 commented on a change in pull request #7185: URL: https://github.com/apache/tvm/pull/7185#discussion_r551170406 ## File path: python/tvm/auto_scheduler/measure.py ## @@ -624,12 +652,10 @@ def local_build_worker(args): The build result of this Builder thread. """ inp, build_func, timeout, verbose = args -if build_func == "default": -build_func = tar.tar -elif build_func == "ndk": -build_func = ndk.create_shared -else: -raise ValueError("Invalid build_func" + build_func) +assert build_func == BuildFunc.name, ( +"BuildFunc.name: " + BuildFunc.name + ", but args is: " + build_func +) +build_func = BuildFunc.build_func Review comment: @jcf94 I try use @tvm._ffi.register_func, but the custom build func is a python callable, it is not just function. we need the whole python callable context, This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [tvm] leowang1225 commented on a change in pull request #7185: [AutoScheduler] Add custom build function
leowang1225 commented on a change in pull request #7185: URL: https://github.com/apache/tvm/pull/7185#discussion_r551162129 ## File path: python/tvm/auto_scheduler/measure.py ## @@ -624,12 +647,7 @@ def local_build_worker(args): The build result of this Builder thread. """ inp, build_func, timeout, verbose = args -if build_func == "default": -build_func = tar.tar -elif build_func == "ndk": -build_func = ndk.create_shared -else: -raise ValueError("Invalid build_func" + build_func) +build_func = BuildFunc.build_func Review comment: custom build func is a python callable object, and it has attribute and so on, we can not serialize it into args. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [tvm] leowang1225 commented on a change in pull request #7185: [AutoScheduler] Add custom build function
leowang1225 commented on a change in pull request #7185: URL: https://github.com/apache/tvm/pull/7185#discussion_r551153793 ## File path: python/tvm/auto_scheduler/measure.py ## @@ -652,7 +652,6 @@ def local_build_worker(args): The build result of this Builder thread. """ inp, build_func, timeout, verbose = args -assert any(build_func == name for name in BuildFunc.name) Review comment: any is error, so I change it This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [tvm] leowang1225 commented on a change in pull request #7185: [AutoScheduler] Add custom build function
leowang1225 commented on a change in pull request #7185: URL: https://github.com/apache/tvm/pull/7185#discussion_r551140443 ## File path: python/tvm/auto_scheduler/measure.py ## @@ -303,12 +312,26 @@ class LocalBuilder(ProgramBuilder): This is used in a wrapper of the multiprocessing.Process.join(). n_parallel : int = multiprocessing.cpu_count() Number of threads used to build in parallel. -build_func : str = 'default' -The name of registered build function. +build_func: callable or str = "default" +If is 'default', use default build function +If is 'ndk', use function for android ndk +If is callable, use it as custom build function, expect lib_format field. """ def __init__(self, timeout=15, n_parallel=multiprocessing.cpu_count(), build_func="default"): -self.__init_handle_by_constructor__(_ffi_api.LocalBuilder, timeout, n_parallel, build_func) +if build_func == "default": Review comment: when auto_scheduler.local_builder.build is called, the context can't get LocalBuilder class instance This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [tvm] leowang1225 commented on a change in pull request #7185: [AutoScheduler] Add custom build function
leowang1225 commented on a change in pull request #7185: URL: https://github.com/apache/tvm/pull/7185#discussion_r551135097 ## File path: python/tvm/auto_scheduler/measure.py ## @@ -63,6 +63,9 @@ # We use 1e10 instead of sys.float_info.max for better readability in log MAX_FLOAT = 1e10 +class CustomBuildFunc: Review comment: done This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [tvm] leowang1225 commented on a change in pull request #7185: [AutoScheduler] Add custom build function
leowang1225 commented on a change in pull request #7185: URL: https://github.com/apache/tvm/pull/7185#discussion_r551133295 ## File path: python/tvm/auto_scheduler/measure.py ## @@ -303,11 +306,16 @@ class LocalBuilder(ProgramBuilder): This is used in a wrapper of the multiprocessing.Process.join(). n_parallel : int = multiprocessing.cpu_count() Number of threads used to build in parallel. -build_func : str = 'default' -The name of registered build function. +build_func: callable or str Review comment: I have added BuildFunc default name = "default" and default func callable = tar.tar This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [tvm] leowang1225 commented on a change in pull request #7185: [AutoScheduler] Add custom build function
leowang1225 commented on a change in pull request #7185: URL: https://github.com/apache/tvm/pull/7185#discussion_r551132967 ## File path: python/tvm/auto_scheduler/measure.py ## @@ -628,6 +636,8 @@ def local_build_worker(args): build_func = tar.tar elif build_func == "ndk": build_func = ndk.create_shared +elif build_func == "custom": Review comment: I change all conditions to BuildFunc This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org