Re: [Tutor] Collating date data from a csv file
Cameron Simpson wrote: > On 08May2019 21:04, Dave Hill wrote: >>I have a csv file which details the results of equipment tests, I >>carry out PAT testing as a volunteer at a heriatge railway in N. >>Wales. I want to extract how many items were tested on each test day. >>So far I have generated a List of test dates, but I am now stalled at >>how to efficiently count numbers tested on each date. >> >>Can I have a list of tuples, where one item is the date and the second >>the count? > > Not as such, because you can't modify a tuple (so you can't update the > count part). But you could use a 2 element list. > >>or is there a better construct? > > Oh definitely. The easiest thing would be a defaultdict(int). Example: > > from collections import defaultdict > ... > by_date = defaultdict(int) > for row in csvdata: > timestamp = row[1] # based on your example data > # get the date from the timestamp > date = ... > by_date[date] += 1 > > A defaultdict is a dict which magicly makes missing elements when they > get access, using a factory function you supply. Here we're using "int" > as that factory, as int() returns zero. While this is easily adaptable if you want to keep more data... by_date = defaultdict(list) # rows grouped by date for row in csvdata: date = ... by_date[date].append(row) ... for the simple case there is also collections.Counter: def get_date(row): return datetime.datetime.fromtimestamp(int(row[1])).date() by_date = collections.Counter(map(get_date, csvdata)) # (date, freq) pairs ordered by frequency: print(by_date.most_common()) > > I presume you've got the timestamp => date conversion sorted? > > Cheers, > Cameron Simpson > ___ > Tutor maillist - Tutor@python.org > To unsubscribe or change subscription options: > https://mail.python.org/mailman/listinfo/tutor ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
[Tutor] Consequences of removing python3-venv
My Linux mint18 system recently wanted to up date python 3.6. When I clicked install I got the following message: The following 2 packages will be removed: python3-dev python3-venv I have 2 virtual environments installed: python 3.5.2 & 3.6.7. Listing the various pythons I have installed shows: jfb@jims-mint18 ~ $ ls -ls /usr/bin/python* 0 lrwxrwxrwx 1 root root 9 Nov 23 2017 /usr/bin/python -> python2.7 0 lrwxrwxrwx 1 root root 9 Nov 23 2017 /usr/bin/python2 -> python2.7 3412 -rwxr-xr-x 1 root root 3492656 Nov 12 13:46 /usr/bin/python2.7 0 lrwxrwxrwx 1 root root 9 Jan 9 2017 /usr/bin/python3 -> python3.5 4360 -rwxr-xr-x 2 root root 4464368 Nov 12 10:27 /usr/bin/python3.5 0 lrwxrwxrwx 1 root root 33 Nov 12 10:27 /usr/bin/python3.5-config -> x86_64-linux-gnu-python3.5-config 4360 -rwxr-xr-x 2 root root 4464368 Nov 12 10:27 /usr/bin/python3.5m 0 lrwxrwxrwx 1 root root 34 Nov 12 10:27 /usr/bin/python3.5m-config -> x86_64-linux-gnu-python3.5m-config 4500 -rwxr-xr-x 2 root root 4604416 Oct 25 2018 /usr/bin/python3.6 0 lrwxrwxrwx 1 root root 33 Oct 25 2018 /usr/bin/python3.6-config -> x86_64-linux-gnu-python3.6-config 4500 -rwxr-xr-x 2 root root 4604416 Oct 25 2018 /usr/bin/python3.6m 0 lrwxrwxrwx 1 root root 34 Oct 25 2018 /usr/bin/python3.6m-config -> x86_64-linux-gnu-python3.6m-config 0 lrwxrwxrwx 1 root root 16 Mar 23 2016 /usr/bin/python3-config -> python3.5-config 0 lrwxrwxrwx 1 root root 10 Jan 9 2017 /usr/bin/python3m -> python3.5m 0 lrwxrwxrwx 1 root root 17 Mar 23 2016 /usr/bin/python3m-config -> python3.5m-config So will allowing the update harm my virtual environments? I really don't want to reinstall them. thanks, Jim ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Collating date data from a csv file
Thank you, I now have
defaultdict(, {736697: 10, 736677: 14, 736980: 9, 737109:
50, 736919: 15, 736652: 19, 736502: 14, 736710: 2, 736674: 6, 736672: 5,
736933: 2, 736932: 6, 736658: 7, 736671: 5, 736499: 6, 736707: 4,
737181: 4, 736686: 2, ...
where the first number is the proleptic Gregorian ordinal of the date
8-) ( I had to look up what proleptic meant)
This means I can access the elements by the ordinal of the date, for
later processing, and extraction to a spreadsheet
Dave
On 09/05/2019 04:08, Cameron Simpson wrote:
On 08May2019 21:04, Dave Hill wrote:
I have a csv file which details the results of equipment tests, I
carry out PAT testing as a volunteer at a heriatge railway in N.
Wales. I want to extract how many items were tested on each test day.
So far I have generated a List of test dates, but I am now stalled at
how to efficiently count numbers tested on each date.
Can I have a list of tuples, where one item is the date and the
second the count?
Not as such, because you can't modify a tuple (so you can't update the
count part). But you could use a 2 element list.
or is there a better construct?
Oh definitely. The easiest thing would be a defaultdict(int). Example:
from collections import defaultdict
...
by_date = defaultdict(int)
for row in csvdata:
timestamp = row[1] # based on your example data
# get the date from the timestamp
date = ...
by_date[date] += 1
A defaultdict is a dict which magicly makes missing elements when they
get access, using a factory function you supply. Here we're using
"int" as that factory, as int() returns zero.
I presume you've got the timestamp => date conversion sorted?
Cheers,
Cameron Simpson
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Collating date data from a csv file
On 09/05/2019 14:06, Dave Hill wrote:
> Thank you, I now have
>
> defaultdict(, {736697: 10, 736677: 14, 736980: 9, 737109:
> 50, 736919: 15, 736652: 19, 736502: 14, 736710: 2, 736674: 6, 736672: 5,
> 736933: 2, 736932: 6, 736658: 7, 736671: 5, 736499: 6, 736707: 4,
> 737181: 4, 736686: 2, ...
>
> where the first number is the proleptic Gregorian ordinal of the date
> 8-) ( I had to look up what proleptic meant)
>
> This means I can access the elements by the ordinal of the date, for
> later processing, and extraction to a spreadsheet
It might be easier to use a DateTime object instead of the ordinal.
It might make printing the results easier later on. You'll find
it in the datetime module...
--
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Consequences of removing python3-venv
On 5/9/19 9:52 AM, Jim wrote: > My Linux mint18 system recently wanted to up date python 3.6. When I > clicked install I got the following message: > > The following 2 packages will be removed: > python3-dev > python3-venv > > I have 2 virtual environments installed: python 3.5.2 & 3.6.7. > > Listing the various pythons I have installed shows: > > jfb@jims-mint18 ~ $ ls -ls /usr/bin/python* > 0 lrwxrwxrwx 1 root root 9 Nov 23 2017 /usr/bin/python -> > python2.7 > 0 lrwxrwxrwx 1 root root 9 Nov 23 2017 /usr/bin/python2 -> > python2.7 > 3412 -rwxr-xr-x 1 root root 3492656 Nov 12 13:46 /usr/bin/python2.7 > 0 lrwxrwxrwx 1 root root 9 Jan 9 2017 /usr/bin/python3 -> > python3.5 > 4360 -rwxr-xr-x 2 root root 4464368 Nov 12 10:27 /usr/bin/python3.5 > 0 lrwxrwxrwx 1 root root 33 Nov 12 10:27 > /usr/bin/python3.5-config -> x86_64-linux-gnu-python3.5-config > 4360 -rwxr-xr-x 2 root root 4464368 Nov 12 10:27 /usr/bin/python3.5m > 0 lrwxrwxrwx 1 root root 34 Nov 12 10:27 > /usr/bin/python3.5m-config -> x86_64-linux-gnu-python3.5m-config > 4500 -rwxr-xr-x 2 root root 4604416 Oct 25 2018 /usr/bin/python3.6 > 0 lrwxrwxrwx 1 root root 33 Oct 25 2018 > /usr/bin/python3.6-config -> x86_64-linux-gnu-python3.6-config > 4500 -rwxr-xr-x 2 root root 4604416 Oct 25 2018 /usr/bin/python3.6m > 0 lrwxrwxrwx 1 root root 34 Oct 25 2018 > /usr/bin/python3.6m-config -> x86_64-linux-gnu-python3.6m-config > 0 lrwxrwxrwx 1 root root 16 Mar 23 2016 /usr/bin/python3-config > -> python3.5-config > 0 lrwxrwxrwx 1 root root 10 Jan 9 2017 /usr/bin/python3m -> > python3.5m > 0 lrwxrwxrwx 1 root root 17 Mar 23 2016 > /usr/bin/python3m-config -> python3.5m-config > > So will allowing the update harm my virtual environments? > > I really don't want to reinstall them. venv shouldn't need to be a separate package, since it's considered "part of Python3", but I don't know what they're thinking exactly over on the debian side. Maybe it's only for the command, and the module is part of the regular distribution? My pyvenv script says not to use it: WARNING: the pyenv script is deprecated in favour of `python3.7 -m venv` In any case, I believe python3-venv is a suitable virtual package, such that it makes sure the one for the right Python exists - as long as it's not trying to remove python3-venv you should be okay. Or, maybe Mint have decided to actually remove the command and you'll have to fall back to calling it as a module as noted above. Removing the command won't remove your virtualenvs. But, the Python upgrade may cause some of your virtual environments might break, and you'll have to refresh them - if they point to, rather than duplicate, system stuff which is changing version. There's a --upgrade option. So tread somewhat carefully. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
