Re: [Matplotlib-users] Best way to use Excel Data
Hi, a strong advice from someone who is using excel format with tons of data is to save them in csv and then import in Sqlite. Excel messes up the data types and gives a lot of troubles with numbers. Sqlite is fast and data are secure. The power of this system is that you can query your data and plot what you need for example. I can share with you my script to import from csv to sqlite if you want. Jahan Mohiuddin wrote: > > Hi, > > I am an novice-intermediate user of python (I took a 1 semester course > in scientific programming with python). I wanted to know what the > best way is to manipulate, analyze, and plot Microsoft Excel data. > The methods I've looked into: > > 1. Save data in CSV file and use csv.dictreader to create a > dictionary. I actually couldn't figure this one out. > > 2. Read the CSV and extract the numerical data into an array of > floats so I can manipulate the data. This is alright but then I lose > the text for the header row. > > > > I have a fairly large biological data set, but I also really just want > to practice with a good method. Any advice is much appreciated. > Thanks! > > > > -Jahan > > > Jahan Mohiuddin > University of North Carolina at Chapel Hill, 2010 > Cell: (516) 480-4825 > > > > > > -- > Beautiful is writing same markup. Internet Explorer 9 supports > standards for HTML5, CSS3, SVG 1.1, ECMAScript5, and DOM L2 & L3. > Spend less time writing and rewriting code and more time creating great > experiences on the web. Be a part of the beta today. > http://p.sf.net/sfu/beautyoftheweb > ___ > Matplotlib-users mailing list > Matplotlib-users@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/matplotlib-users > > -- View this message in context: http://old.nabble.com/Best-way-to-use-Excel-Data-tp29908079p29926020.html Sent from the matplotlib - users mailing list archive at Nabble.com. -- Beautiful is writing same markup. Internet Explorer 9 supports standards for HTML5, CSS3, SVG 1.1, ECMAScript5, and DOM L2 & L3. Spend less time writing and rewriting code and more time creating great experiences on the web. Be a part of the beta today. http://p.sf.net/sfu/beautyoftheweb ___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
Re: [Matplotlib-users] Scatter Plot with different colors
2010/10/10 Alessio Civ : > > > Please, can someone help me? I've been digging the documentation, but I > can't find a way to do this. ¿Didn't you get my message on oct-5? I didn't send it to the list by mistake: --%< Make your variables numpy arrays and slice them using values in z: x = np.array([1, 2, 3, 4]) y = np.array([2, 3, 4, 5]) z = np.array([0, 1, 0, 1]) x0 = x[z == 0] y0 = y[z == 1] x1 = x[z == 0] y1 = y[z == 1] plt.scatter(x0, y0, c='b') plt.scatter(x1, y1, c='r') --%< See the attached example, you can run it as a script or import it as a module and use the function multi_scatter in your code. Goyo multi_scatter.py Description: Binary data -- Beautiful is writing same markup. Internet Explorer 9 supports standards for HTML5, CSS3, SVG 1.1, ECMAScript5, and DOM L2 & L3. Spend less time writing and rewriting code and more time creating great experiences on the web. Be a part of the beta today. http://p.sf.net/sfu/beautyoftheweb___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
Re: [Matplotlib-users] Best way to use Excel Data
Hi, To load csv data, I use a modified version of csv2rec for which the data type of each column is specified explicitly in the data file. By removing the dtype guessing you get a speedup and you also avoid potential mess-ups. Alessio: sadly you right about it not being possible to trust Excel with data. Could you please give more details on the sqlite method you suggest ? -- >> by Alessio Civ Oct 10, 2010; 09:04am: Hi, a strong advice from someone who is using excel format with tons of data is to save them in csv and then import in Sqlite. Excel messes up the data types and gives a lot of troubles with numbers. Sqlite is fast and data are secure. The power of this system is that you can query your data and plot what you need for example. I can share with you my script to import from csv to sqlite if you want. -- thanks, peter butterworth -- Beautiful is writing same markup. Internet Explorer 9 supports standards for HTML5, CSS3, SVG 1.1, ECMAScript5, and DOM L2 & L3. Spend less time writing and rewriting code and more time creating great experiences on the web. Be a part of the beta today. http://p.sf.net/sfu/beautyoftheweb ___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
Re: [Matplotlib-users] Best way to use Excel Data
Hi Butterw, first of all, if you are working on data a lot, we could get in contact. I need to work better on my scripts and we could help each other. I've uploaded my script that imports from csv to sqlite. It's not a totally clean script, as I am working on something else now I couldn't clean what I did. Anyway, you should get an idea of what it does. Basically, the script cleans up the messed data from excel (take out the comas, puts point, convert numbers to floats). Then, the scripts writes the resulting tuple in an sqlite database. This script import over 20 records in around 3 seconds. After this, you can access your data connecting to sqlite. Accessing sqlite data is extremely easy and efficient. I'm uploading a small script to plot from Sqlite and getting the values for a linear regression, it just works fine and smoothly. Let me know if are interested in helping each other. http://old.nabble.com/file/p29928938/SqlitePlot_GDF.py SqlitePlot_GDF.py http://old.nabble.com/file/p29928938/SqliteImportCsv.py SqliteImportCsv.py butterw wrote: > > Hi, > > > To load csv data, I use a modified version of csv2rec for which the > data type of each column is specified explicitly in the data file. > By removing the dtype guessing you get a speedup and you also avoid > potential mess-ups. > > > Alessio: sadly you right about it not being possible to trust Excel with > data. > Could you please give more details on the sqlite method you suggest ? > > > -- >>> by Alessio Civ Oct 10, 2010; 09:04am: > Hi, > > a strong advice from someone who is using excel format with tons of > data is to save them in csv and then import in Sqlite. > > Excel messes up the data types and gives a lot of troubles with > numbers. Sqlite is fast and data are secure. > The power of this system is that you can query your data and plot what > you need for example. > > I can share with you my script to import from csv to sqlite if you want. > > > -- > thanks, > peter butterworth > > -- > Beautiful is writing same markup. Internet Explorer 9 supports > standards for HTML5, CSS3, SVG 1.1, ECMAScript5, and DOM L2 & L3. > Spend less time writing and rewriting code and more time creating great > experiences on the web. Be a part of the beta today. > http://p.sf.net/sfu/beautyoftheweb > ___ > Matplotlib-users mailing list > Matplotlib-users@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/matplotlib-users > > -- View this message in context: http://old.nabble.com/Best-way-to-use-Excel-Data-tp29908079p29928938.html Sent from the matplotlib - users mailing list archive at Nabble.com. -- Beautiful is writing same markup. Internet Explorer 9 supports standards for HTML5, CSS3, SVG 1.1, ECMAScript5, and DOM L2 & L3. Spend less time writing and rewriting code and more time creating great experiences on the web. Be a part of the beta today. http://p.sf.net/sfu/beautyoftheweb ___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
Re: [Matplotlib-users] Scatter Plot with different colors
Hi,
thanks for reply.
My problem is that I need to scatter data of different periods but in the
same x.
For example, the quantity x sold in March is 1 at price y of 2, the quantity
sold in April is 2 at price y of 3. I want the 2 points to have different
colors in the same graph!
Now I'm doing:
for i in list:
x.append(i[0])
y.append(i[1])
axScatter.scatter(x, y, c='r', marker='s')
This makes al the points red! I want to be able to put points of different
colors for different periods of time. Basically, I want to know how to
define a function in order to change the color of the points for specific
definitions.
Jae-Joon Lee wrote:
>
> On Tue, Oct 5, 2010 at 11:07 PM, Alessio Civ
> wrote:
>>
>> Hi,
>>
>> I'm trying to make a scatter plot of 2 variables using a thirds as filter
>> to
>> have different colors.
>>
>> Let's say I have those data:
>>
>> x=1,2,3,4
>> y=2,3,4,5
>> z=0,1,0,1
>>
>> Then I want the values of x and y corresponding to those of z=0 to be of
>> a
>> color and those corresponding to z=1 to be of another color.
>>
>> This is the code I manage to do until know:
>>
>>
>> import xlrd
>> import numpy as np
>> import matplotlib.pyplot as plt
>>
>> wb = xlrd.open_workbook('GBL2009.xls')
>> sh = wb.sheet_by_index(0)
>>
>> def column_pos():
>> first_row=sh.row_values(0)
>> net_p=""
>> for i in first_row:
>> if i=='net_price':
>> net_p=first_row.index(i) #In gets the column position
>> for i in first_row:
>> if i=='material':
>> mat_p=first_row.index(i) #In gets the column position
>> for i in first_row:
>> if i=='qty':
>> qty_p=first_row.index(i) #In gets the column position
>> print net_p, mat_p, qty_p
>> #filtering(net_p, mat_p, qty_p)
>> test(net_p, mat_p, qty_p)
>>
>>
>> def test(net_p, mat_p, qty_p):
>> list=[]
>> for rownum in range(sh.nrows):
>> if sh.cell(rownum,mat_p).value in (96433890, 96433886):
>> list.append(sh.row_values(rownum))
>>
>> x=[]
>> y=[]
>> z=[]
>> for i in list:
>> x.append(i[qty_p])
>> y.append(i[net_p])
>> z.append(i[mat_p])
>>
>> fig = plt.figure(1, figsize=(5.5,5.5))
>> axScatter = plt.subplot(111)
>>
>> colors = ('r', 'g', 'b', 'k')
>> for c in colors:
>> axScatter.scatter(x, y, c=c, marker='s')
>>
>> plt.show()
>>
>
> So, what is your problem?
> If you want to post a code, please post a complete (but simple!) code.
>
> If you want to map values of z to colors, a simple solution would be
> to use dict.
>
> z = [0,1,0,1]
> color_map = {0:"r", 1:"g"}
> z_as_colors = map(color_map.get, z)
>
> scatter supports colormap but it may not very useful for your case.
>
> Regards,
>
> -JJ
>
> --
> Beautiful is writing same markup. Internet Explorer 9 supports
> standards for HTML5, CSS3, SVG 1.1, ECMAScript5, and DOM L2 & L3.
> Spend less time writing and rewriting code and more time creating great
> experiences on the web. Be a part of the beta today.
> http://p.sf.net/sfu/beautyoftheweb
> ___
> Matplotlib-users mailing list
> Matplotlib-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/matplotlib-users
>
>
--
View this message in context:
http://old.nabble.com/Scatter-Plot-with-different-colors-tp29887701p29928970.html
Sent from the matplotlib - users mailing list archive at Nabble.com.
--
Beautiful is writing same markup. Internet Explorer 9 supports
standards for HTML5, CSS3, SVG 1.1, ECMAScript5, and DOM L2 & L3.
Spend less time writing and rewriting code and more time creating great
experiences on the web. Be a part of the beta today.
http://p.sf.net/sfu/beautyoftheweb
___
Matplotlib-users mailing list
Matplotlib-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/matplotlib-users
Re: [Matplotlib-users] Scatter Plot with different colors
Thanks Goyo and Jae-Joon Lee. It worked! it worked! Goyo wrote: > > 2010/10/10 Alessio Civ : >> >> >> Please, can someone help me? I've been digging the documentation, but I >> can't find a way to do this. > > ¿Didn't you get my message on oct-5? I didn't send it to the list by > mistake: > > --%< > Make your variables numpy arrays and slice them using values in z: > > x = np.array([1, 2, 3, 4]) > y = np.array([2, 3, 4, 5]) > z = np.array([0, 1, 0, 1]) > > x0 = x[z == 0] > y0 = y[z == 1] > x1 = x[z == 0] > y1 = y[z == 1] > > plt.scatter(x0, y0, c='b') > plt.scatter(x1, y1, c='r') > > --%< > > See the attached example, you can run it as a script or import it as a > module and use the function multi_scatter in your code. > > Goyo > > > -- > Beautiful is writing same markup. Internet Explorer 9 supports > standards for HTML5, CSS3, SVG 1.1, ECMAScript5, and DOM L2 & L3. > Spend less time writing and rewriting code and more time creating great > experiences on the web. Be a part of the beta today. > http://p.sf.net/sfu/beautyoftheweb > ___ > Matplotlib-users mailing list > Matplotlib-users@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/matplotlib-users > > -- View this message in context: http://old.nabble.com/Scatter-Plot-with-different-colors-tp29887701p29931461.html Sent from the matplotlib - users mailing list archive at Nabble.com. -- Beautiful is writing same markup. Internet Explorer 9 supports standards for HTML5, CSS3, SVG 1.1, ECMAScript5, and DOM L2 & L3. Spend less time writing and rewriting code and more time creating great experiences on the web. Be a part of the beta today. http://p.sf.net/sfu/beautyoftheweb ___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
