#3639: Write table like output directly to database --------------------------+---------------------------------- Reporter: sbl | Owner: grass-dev@… Type: enhancement | Status: new Priority: normal | Milestone: 8.0.0 Component: Database | Version: svn-trunk Resolution: | Keywords: table,database,write CPU: Unspecified | Platform: Unspecified --------------------------+----------------------------------
Comment (by sbl): I am considering adding a function to the Python scripting library for parsing table-like stdout from GRASS modules into Numpy arrays. Nothing complicated, but probably a convenient wrapper function!? See: {{{ def stdout2numpy(stdout=None, sep=',', names=False, null_value=None, fill_value=None, comments='#', usecols=None): """Read table-like output from grass modules as Numpy array; format instructions are handed down to Numpys genfromtxt function param str|byte stdout: tabular stdout from GRASS GIS module call param str sep: Separator delimiting columns param list names: List of strings with names for columns param str null_value: Characters representing the no-data value param str fill_value: Value to fill no-data with param str comments: Character that param list usecols: List of columns to import """ import numpy as np from io import BytesIO if type(stdout) == str: stdout = gscript.encode(stdout) elif type(stdout) != byte: gscript.fatal(_('Unsupported data type')) np_array = np.genfromtxt(BytesIO(stdout), missing_values=null_value, filling_values=fill_value, usecols=usecols, names=names, dtype=None, delimiter=sep) return np_array }}} or alternatively as np_parse_command (equivalent to parse_command()): {{{ def np_parse_command(*args, **kwargs): """Passes all arguments to read_command, then parses the output using Numpys genfromtxt() function to generate a Numpy array from table like output. Parsing options in Numpys genfromtxt() function can be optionally given by <em>parse</em> dictionary, e.g. :: parse_command(..., parse = { 'delimiter' : '|' })) As far as possible, at least standard parser options in GRASS commands are handed down and translated to genfromtxt() automatically, e.g. separator (GRASS) -> delimiter (Numpy). import numpy as np from io import BytesIO stdout = read_command(*args, **kwargs) if type(stdout) == str: stdout = gscript.encode(stdout) elif type(stdout) != byte: gscript.fatal(_('Unsupported data type')) filling_values = parse['filling_values'] if 'filling_values' in parse else np.nan usecols = parse['usecols'] if 'usecols' in parse else None names = parse['names'] if 'names' in parse else None dtype = parse['dtype'] if 'dtype' in parse else None # Check if separator is specified in GRASS command if 'delimiter' not in parse: if 'separator' in args: parse['delimiter'] = args['separator'] elif 'separator' in kwargs: parse['delimiter'] = kwargs['separator'] else: parse['delimiter'] = ',' # Check if null_value is specified in GRASS command if 'missing_values' not in parse: if 'null_value' in args: parse['missing_values'] = args['null_value'] elif 'null_value' in kwargs: parse['missing_values'] = kwargs['null_value'] else: parse['missing_values'] = '*' np_array = np.genfromtxt(BytesIO(stdout), **parse) return np_array }}} That would probably be a first step for a Python function that can write output to DB... Any thoughts/opinions? -- Ticket URL: <https://trac.osgeo.org/grass/ticket/3639#comment:2> GRASS GIS <https://grass.osgeo.org>
_______________________________________________ grass-dev mailing list grass-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/grass-dev