write to a file two dict()
Hi
Have two dict() of the same length and i want print them to a common file.
a={1: 1, 2: 2, 3: 3}
b={1: 11, 2: 22, 3: 33}
in order to obtain
1 1 1 11
2 2 2 22
3 3 3 33
I tried
output = open(dst_file, w)
for (a), b , (c) , d in a.items() , b.items() :
output.write(%i %i %i %i\n % (a,b,c,d))
output.close()
but i get the error ValueError: need more than 3 values to unpack.
do you have some suggestions?.
Thanks
Giuseppe
--
http://mail.python.org/mailman/listinfo/python-list
looping in array vs looping in a dic
Hi, I have this script in python that i need to apply for very large arrays (arrays coming from satellite images). The script works grate but i would like to speed up the process. The larger computational time is in the for loop process. Is there is a way to improve that part? Should be better to use dic() instead of np.ndarray for saving the results? and if yes how i can make the sum in dic()(like in the correspondent matrix[row_c,1] = matrix[row_c,1] + valuesRaster[row,col] )? If the dic() is the solution way is faster? Thanks Giuseppe import numpy as np import sys from time import clock, time # create the arrays start = time() valuesRaster = np.random.random_integers(0, 100, 100).reshape(10, 10) valuesCategory = np.random.random_integers(1, 10, 100).reshape(10, 10) elapsed = (time() - start) print(elapsed , create the data) start = time() categories = np.unique(valuesCategory) matrix = np.c_[ categories , np.zeros(len(categories))] elapsed = (time() - start) print(elapsed , create the matrix and append a colum zero ) rows = 10 cols = 10 start = time() for col in range(0,cols): for row in range(0,rows): for row_c in range(0,len(matrix)) : if valuesCategory[row,col] == matrix[row_c,0] : matrix[row_c,1] = matrix[row_c,1] + valuesRaster[row,col] break elapsed = (time() - start) print(elapsed , loop in the data ) print (matrix) -- http://mail.python.org/mailman/listinfo/python-list
Re: looping in array vs looping in a dic
Hi Ian and MRAB thanks to you input i have improve the speed of my code. Definitely reading in dic() is faster. I have one more question. In the dic() I calculate the sum of the values, but i want count also the number of observation, in order to calculate the average in the end. Should i create a new dic() or is possible to do in the same dic(). Here in the final code. Thanks Giuseppe rows = dsCategory.RasterYSize cols = dsCategory.RasterXSize print(Generating output file %s %(dst_file)) start = time() unique=dict() for irows in xrange(rows): valuesRaster=dsRaster.GetRasterBand(1).ReadAsArray(0,irows,cols,1) valuesCategory=dsCategory.GetRasterBand(1).ReadAsArray(0,irows,cols,1) for icols in xrange(cols): if ( valuesRaster[0,icols] != no_data_Raster ) and ( valuesCategory[0,icols] != no_data_Category ) : row = valuesCategory[0, icols],valuesRaster[0, icols] if row[0] in unique : unique[row[0]] += row[1] else: unique[row[0]] = 0+row[1] # this 0 was add if not the first observation was considered = 0 -- http://mail.python.org/mailman/listinfo/python-list
Re: no data exclution and unique combination.
Terry and MRAB,
thanks for yours suggestions,
in the end i found this solution
mask=( a != 0 ) ( b != 0 )
a_mask=a[mask]
b_mask=b[mask]
array2D = np.array(zip(a_mask,b_mask))
unique=dict()
for row in array2D :
row = tuple(row)
if row in unique:
unique[row] += 1
else:
unique[row] = 1
print unique
{(4, 5): 1, (5, 4): 1, (4, 4): 2, (2, 3): 1, (4, 3): 2}
I choose this solution because i could not install from collections import
Counter.
Anyway how i can print to a file the unique results without the brackets and
obtain something like this?
4 5 1
5 4 1
4 4 2
2 3 1
4 3 2
Thanks in advance
Best regards.
Giuseppe
On Tuesday, 24 July 2012 13:27:05 UTC-5, giuseppe...@gmail.com wrote:
Hi,
would like to take eliminate a specific number in an array and its
correspondent in an other array, and vice-versa.
given
a=np.array([1,2,4,4,5,4,1,4,1,1,2,4])
b=np.array([1,2,3,5,4,4,1,3,2,1,3,4])
no_data_a=1
no_data_b=2
a_clean=array([4,4,5,4,4,4])
b_clean=array([3,5,4,4,3,4])
after i need to calculate unique combination in pairs to count the
observations
and obtain
(4,3,2)
(4,5,1)
(5,4,1)
(4,4,2)
For the fist task i did
a_No_data_a = a[a != no_data_a]
b_No_data_a = b[a != no_data_a]
b_clean = b_No_data_a[b_No_data_a != no_data_b]
a_clean = a_No_data_a[a_No_data_a != no_data_b]
but the results are not really stable.
For the second task
The np.unique would solve the problem if it can be apply to a two arrays.
Any idea?
thanks in advance
Giuseppe
--
http://mail.python.org/mailman/listinfo/python-list
save dictionary to a file without brackets.
Hi,
I have a dict() unique
like this
{(4, 5): 1, (5, 4): 1, (4, 4): 2, (2, 3): 1, (4, 3): 2}
and i want to print to a file without the brackets comas and semicolon in order
to obtain something like this?
4 5 1
5 4 1
4 4 2
2 3 1
4 3 2
Any ideas?
Thanks in advance
Giuseppe
--
http://mail.python.org/mailman/listinfo/python-list
Re: save dictionary to a file without brackets.
thanks for the fast replies
my testing were very closed to yours but i did not know how
On 9 August 2012 15:25, Oscar Benjamin oscar.j.benja...@gmail.com wrote:
On Aug 9, 2012 9:17 PM, giuseppe.amatu...@gmail.com wrote:
Hi,
I have a dict() unique
like this
{(4, 5): 1, (5, 4): 1, (4, 4): 2, (2, 3): 1, (4, 3): 2}
and i want to print to a file without the brackets comas and semicolon in
order to obtain something like this?
4 5 1
5 4 1
4 4 2
2 3 1
4 3 2
Any ideas?
Thanks in advance
How's this?
from __future__ import print_function
output = open(out.txt, w)
for (a, b), c in d.items():
print(a, b, c, file=output)
output.close()
Oscar.
--
http://mail.python.org/mailman/listinfo/python-list
--
Giuseppe Amatulli
Web: www.spatial-ecology.net
--
http://mail.python.org/mailman/listinfo/python-list
Re: save dictionary to a file without brackets.
thanks for the fast replies
my testing were very closed to yours but i did not know how to print
the the number after the semicolon!
thanks!
On 9 August 2012 15:25, Oscar Benjamin oscar.j.benja...@gmail.com wrote:
On Aug 9, 2012 9:17 PM, giuseppe.amatu...@gmail.com wrote:
Hi,
I have a dict() unique
like this
{(4, 5): 1, (5, 4): 1, (4, 4): 2, (2, 3): 1, (4, 3): 2}
and i want to print to a file without the brackets comas and semicolon in
order to obtain something like this?
4 5 1
5 4 1
4 4 2
2 3 1
4 3 2
Any ideas?
Thanks in advance
How's this?
from __future__ import print_function
output = open(out.txt, w)
for (a, b), c in d.items():
print(a, b, c, file=output)
output.close()
Oscar.
--
http://mail.python.org/mailman/listinfo/python-list
--
Giuseppe Amatulli
Web: www.spatial-ecology.net
--
http://mail.python.org/mailman/listinfo/python-list
Re: save dictionary to a file without brackets.
Thanks a lot for the clarification.
Actually my problem is giving to raster dataset in geo-tif format find out
unique pair combination, count the number of observation
unique combination in rast1, count the number of observation
unique combination in rast2, count the number of observation
I try different solution and this seems to me the faster
Rast00=dsRast00.GetRasterBand(1).ReadAsArray()
Rast10=dsRast10.GetRasterBand(1).ReadAsArray()
mask=( Rast00 != 0 ) ( Rast10 != 0 ) # may be this masking
operation can be included in the for loop
Rast00_mask= Rast00[mask]# may be this masking
operation can be included in the for loop
Rast10_mask= Rast10[mask]# may be this masking
operation can be included in the for loop
array2D = np.array(zip( Rast00_mask,Rast10_mask))
unique_u=dict()
unique_k1=dict()
unique_k2=dict()
for key1,key2 in array2D :
row = tuple((key1,key2))
if row in unique_u:
unique_u[row] += 1
else:
unique_u[row] = 1
if key1 in unique_k1:
unique_k1[key1] += 1
else:
unique_k1[key1] = 1
if key2 in unique_k2:
unique_k2[key2] += 1
else:
unique_k2[key2] = 1
output = open(dst_file_rast0010, w)
for (a, b), c in unique_u.items():
print(a, b, c, file=output)
output.close()
output = open(dst_file_rast00, w)
for (a), b in unique_k1.items():
print(a, b, file=output)
output.close()
output = open(dst_file_rast10, w)
for (a), b in unique_k2.items():
print(a, b, file=output)
output.close()
What do you think? is there a way to speed up the process?
Thanks
Giuseppe
On 9 August 2012 16:34, Roman Vashkevich vashkevic...@gmail.com wrote:
Actually, they are different.
Put a dict.{iter}items() in an O(k^N) algorithm and make it a hundred
thousand entries, and you will feel the difference.
Dict uses hashing to get a value from the dict and this is why it's O(1).
10.08.2012, в 1:21, Tim Chase написал(а):
On 08/09/12 15:41, Roman Vashkevich wrote:
10.08.2012, в 0:35, Tim Chase написал(а):
On 08/09/12 15:22, Roman Vashkevich wrote:
{(4, 5): 1, (5, 4): 1, (4, 4): 2, (2, 3): 1, (4, 3): 2}
and i want to print to a file without the brackets comas and semicolon
in order to obtain something like this?
4 5 1
5 4 1
4 4 2
2 3 1
4 3 2
for key in dict:
print key[0], key[1], dict[key]
This might read more cleanly with tuple unpacking:
for (edge1, edge2), cost in d.iteritems(): # or .items()
print edge1, edge2, cost
(I'm making the assumption that this is a edge/cost graph...use
appropriate names according to what they actually mean)
dict.items() is a list - linear access time whereas with 'for
key in dict:' access time is constant:
http://python.net/~goodger/projects/pycon/2007/idiomatic/handout.html#use-in-where-possible-1
That link doesn't actually discuss dict.{iter}items()
Both are O(N) because you have to touch each item in the dict--you
can't iterate over N entries in less than O(N) time. For small
data-sets, building the list and then iterating over it may be
faster faster; for larger data-sets, the cost of building the list
overshadows the (minor) overhead of a generator. Either way, the
iterate-and-fetch-the-associated-value of .items() .iteritems()
can (should?) be optimized in Python's internals to the point I
wouldn't think twice about using the more readable version.
-tkc
--
Giuseppe Amatulli
Web: www.spatial-ecology.net
--
http://mail.python.org/mailman/listinfo/python-list
no data exclution and unique combination.
Hi, would like to take eliminate a specific number in an array and its correspondent in an other array, and vice-versa. given a=np.array([1,2,4,4,5,4,1,4,1,1,2,4]) b=np.array([1,2,3,5,4,4,1,3,2,1,3,4]) no_data_a=1 no_data_b=2 a_clean=array([4,4,5,4,4,4]) b_clean=array([3,5,4,4,3,4]) after i need to calculate unique combination in pairs to count the observations and obtain (4,3,2) (4,5,1) (5,4,1) (4,4,2) For the fist task i did a_No_data_a = a[a != no_data_a] b_No_data_a = b[a != no_data_a] b_clean = b_No_data_a[b_No_data_a != no_data_b] a_clean = a_No_data_a[a_No_data_a != no_data_b] but the results are not really stable. For the second task The np.unique would solve the problem if it can be apply to a two arrays. Any idea? thanks in advance Giuseppe -- http://mail.python.org/mailman/listinfo/python-list
