Re: [Tutor] pandas data frame
Hi Peter Thank you much for your help its very appreciated ,bellow is an example of what i need. The main dataframe ( need to be separated into separate dataframes ). The desired dataframes output correctly as i need it to be,Thanks much Main dataframendx V_id Average Mean Peak 0 1 3 2 51 2 2 1 62 3 4 1 83 1 2 2 74 2 3 3 65 3 5 3 46 1 1 1 87 2 2 5 108 3 5 5 99 1 2 5 1010 2 5 5 911 3 4 3 10 Dataframe-1ndx V_id Average Mean Peak 0 1 3 2 53 1 2 2 76 1 1 1 89 1 2 5 10 Dataframe-2ndx V_id Averag Mean Peak 1 2 2 1 64 2 3 3 67 2 2 5 1010 2 5 5 9 Dataframe-3ndx V_id Average Mean Peak 2 3 4 1 85 3 5 3 48 3 5 5 911 3 4 3 10 Bachir On Monday, January 18, 2016 11:25 AM, Peter Otten <__pete...@web.de> wrote: Bachir Bachir wrote: [Bachir, please send your mails to the list, not to me. That way you increase the likelihood to get a good answer] >> On Sunday, January 17, 2016 4:20 PM, Peter Otten <__pete...@web.de> >> wrote: >> Bachir Bachir via Tutor wrote: >>> Hello EverybodyI need to sort a dataframe according to a specific column >>> the create new dataframes according to the sorted columns each new >>> created dataframe should contain the list(set('the sorted element)any >>> help please, i am new in python and pandas thanks muchBachir >> >> Like this? >> >> >>> df >> >> foo bar >> 0 3.3 b >> 1 2.2 b >> 2 7.5 a >> 3 1.1 c >> 4 4.7 a >> >> [5 rows x 2 columns] >> >> >>> df.groupby("bar").sum() >> >> foo >> bar >> a 12.2 >> b 5.5 >> c 1.1 >> >> [3 rows x 1 columns] > Thanks Peter ,This way but i want keep all the table elements only > separate them by a single column I'm sorry, I don't understand what you mean by "separate them by a single column". Can you give an example? What exactly should the sorted version of >> >>> df >> >> foo bar >> 0 3.3 b >> 1 2.2 b >> 2 7.5 a >> 3 1.1 c >> 4 4.7 a >> >> [5 rows x 2 columns] look like? ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
[Tutor] Fwd:
-- Forwarded message -- From: Deepak NnDate: Mon, Jan 18, 2016 at 6:27 PM Subject: Re: To: tutor-ow...@python.org Please provide a python program to run a program (.exe) and get Hash *exactly* as : 160 106 182 190 228 64 68 207 248 109 67 88 41 .The username to be used is admin .The *password* is what to be found out .The hash provided is of the correct password .Mostly the password will be *13 char long *and has a small chance of being all alpha characters .*GOOGLE DOES NOT ALLOW TO SENT .EXE FILE .* Provide the where is change i have to make in the program to run the .exe file that i have in my computer . Since i can't sent the .exe file .i am sending the screen shorts of the program .*The text file of the program is included and few screenshots of the the the result of the program is also included* . ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] str.strip strange result...?
> From: d...@hashcollision.org > Date: Mon, 18 Jan 2016 00:29:41 -0800 > Subject: Re: [Tutor] str.strip strange result...? > To: sjeik_ap...@hotmail.com > CC: __pete...@web.de; tutor@python.org > > > Not sure which one is faster, but in this case I actually find a regex more > > readable (!): > re.sub(r"_1$", "", "V01_1") > > 'V01' > > > Hi Albert-Jan Roskam, > > Here's a minor counterpoint to using regexes here: they're sometimes a > bit too powerful. Yes, certainly: https://xkcd.com/208/ :-) > > In this situation, a regex approach might be troublesome if parts of > the suffix can be interpreted as regular expression meta-characters. > For example, if we were trying to strip out a suffix like > > ".exe" > > then if we were to try to do this with regexes, we might forget that > "." is a meta-character that acts as a wildcard for any single > character. Therefore, a regexp-based solution for the general > suffix-removal problem is complicated because we'd need consider > escaping certain characters. Not that this would be hard, but that > it's yet another detail we have to keep in our heads. Agreed. Apart from backslashes, re.escape could also be really handy. But regexes are scary and intimidating for those who have never seen them before. > Another disadvantage is that, if the suffix is dynamically determined, > then there's an additional cost in "compiling" the regular expression: > building the pattern-matching machinery doesn't come for free. re.compile helps, though it always bothers me that I "need" to put them as a global in the top-level of my module (along with collections.namedtuple).Unless I use a class, where __init__ is a good place. > For those reasons, I think the regexp approach here is a little bit of > overkill. This kind of tradeoff is the sort of thing that reference > documentation will probably not mention. That's why this list is > here, to share the experience of using these systems with other > beginners and learners. Regexes are still very useful: just be aware > that they have sharp edges. One other downside is that it takes only a couple of days/beers before one forgets the regex. But re.VERBOSE and named groups help a LOT. > Good luck to you! ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] str.strip strange result...?
> Not sure which one is faster, but in this case I actually find a regex more > readable (!): re.sub(r"_1$", "", "V01_1") > 'V01' Hi Albert-Jan Roskam, Here's a minor counterpoint to using regexes here: they're sometimes a bit too powerful. In this situation, a regex approach might be troublesome if parts of the suffix can be interpreted as regular expression meta-characters. For example, if we were trying to strip out a suffix like ".exe" then if we were to try to do this with regexes, we might forget that "." is a meta-character that acts as a wildcard for any single character. Therefore, a regexp-based solution for the general suffix-removal problem is complicated because we'd need consider escaping certain characters. Not that this would be hard, but that it's yet another detail we have to keep in our heads. Another disadvantage is that, if the suffix is dynamically determined, then there's an additional cost in "compiling" the regular expression: building the pattern-matching machinery doesn't come for free. For those reasons, I think the regexp approach here is a little bit of overkill. This kind of tradeoff is the sort of thing that reference documentation will probably not mention. That's why this list is here, to share the experience of using these systems with other beginners and learners. Regexes are still very useful: just be aware that they have sharp edges. Good luck to you! ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] if request.method == 'GET': NameError: global name 'request' is not defined
Here is the code snippet: File#FlaskTest2.py from flask import Flask app = Flask(__name__) #Make an app.route() decorator here @app.route("/puppies/", methods = ['GET' , 'POST']) def puppiesFunction(): if request.method == 'GET': return getAllPuppies() elif request.method == 'POST': return makeANewPuppy() File# FlaskTendpointTester2.py import httplib2 import json import sys print "Running Endpoint Tester\n" address = raw_input("Please enter the address of the server you want to access, \n If left blank the connection will be set to ' http://localhost:5000': ") if address == '': address = 'http://localhost:5000' #Making a GET Request print "Making a GET Request for /puppies..." Virtual machine is running on port:5000 and the error trace seen on when the endpoint code is executed on localhost port:5000 Traceback (most recent call last): File "/Users/sudiptomanna/anaconda2/lib/python2.7/site-packages/flask/app.py", line 1836, in __call__ return self.wsgi_app(environ, start_response) File "/Users/sudiptomanna/anaconda2/lib/python2.7/site-packages/flask/app.py", line 1820, in wsgi_app response = self.make_response(self.handle_exception(e)) File "/Users/sudiptomanna/anaconda2/lib/python2.7/site-packages/flask/app.py", line 1403, in handle_exception reraise(exc_type, exc_value, tb) File "/Users/sudiptomanna/anaconda2/lib/python2.7/site-packages/flask/app.py", line 1817, in wsgi_app response = self.full_dispatch_request() File "/Users/sudiptomanna/anaconda2/lib/python2.7/site-packages/flask/app.py", line 1477, in full_dispatch_request rv = self.handle_user_exception(e) File "/Users/sudiptomanna/anaconda2/lib/python2.7/site-packages/flask/app.py", line 1381, in handle_user_exception reraise(exc_type, exc_value, tb) File "/Users/sudiptomanna/anaconda2/lib/python2.7/site-packages/flask/app.py", line 1475, in full_dispatch_request rv = self.dispatch_request() File "/Users/sudiptomanna/anaconda2/lib/python2.7/site-packages/flask/app.py", line 1461, in dispatch_request return self.view_functions[rule.endpoint](**req.view_args) File "/Users/sudiptomanna/fullstack/vagrant/PythonData/FlaskTest2.py", line 8, in puppiesFunction if request.method == 'GET': NameError: global name 'request' is not defined Regards, Sudipto Manna On Sun, Jan 17, 2016 at 7:17 PM, Alan Gauldwrote: > On 17/01/16 23:15, sudipto manna wrote: > > Hi All, > > I am running a python flask project for fetching the endpoint. > > > > Please find the files attached. > > OK, This is a text based mailing list so attachments tend > not to make it through the server. If they are not too long post them > here, or if they are bigger put them on a pastebin. However... > > Flask is not part of the standard library so a little bit > off topic for this list, you may be better off asking on > the Flask support forum. > > Howe er it is a common framework so somebody here might > be able to help. > > > When the endpoint file is executed its returning this error: > > "if request.method == 'GET': NameError: global name 'request' is not > > defined" > > We really need to see the full error trace and the full > context around the reported fault - ideally the full file. > The probability is that the error is telling the truth > and 'request' is not defined yet. Is it part of Flask? > Should you have prepended the module name? > Is the spelling/case correct? > > > -- > Alan G > Author of the Learn to Program web site > http://www.alan-g.me.uk/ > http://www.amazon.com/author/alan_gauld > Follow my photo-blog on Flickr at: > http://www.flickr.com/photos/alangauldphotos > > > ___ > Tutor maillist - Tutor@python.org > To unsubscribe or change subscription options: > https://mail.python.org/mailman/listinfo/tutor > ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] if request.method == 'GET': NameError: global name 'request' is not defined
On Sun, Jan 17, 2016 at 6:13 PM, sudipto mannawrote: > Here is the code snippet: > > File#FlaskTest2.py > > from flask import Flask > > app = Flask(__name__) > > #Make an app.route() decorator here > @app.route("/puppies/", methods = ['GET' , 'POST']) > def puppiesFunction(): > if request.method == 'GET': > return getAllPuppies() Hi Sudipto, I don't know the Flask framework here, but this code looks suspicious. What is "request" in this puppiesFunction definition? It's not a parameter. Where does it come from? Is it a global? http://flask.pocoo.org/docs/0.10/quickstart/#accessing-request-data ... oh. It is a module global. That is not something I like philosophically, but oh well, roll with it... Do you have other working examples from prior code that used requests? If so, you may want to double check them. You're missing a definition that you need to add to your program. Take a look at the documentation: read the "The Request Object" section in the docs, and you should see the missing line that you need to add to your program. If you can't find it after you've read this quickstart documentation, please feel free to ask the mailing list. But you should be able to see it: it's the first boxed piece of code at: http://flask.pocoo.org/docs/0.10/quickstart/#the-request-object Good luck! ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] if request.method == 'GET': NameError: global name 'request' is not defined
On 18/01/16 02:13, sudipto manna wrote: > Here is the code snippet: > > File#FlaskTest2.py > > from flask import Flask > > app = Flask(__name__) > > #Make an app.route() decorator here > @app.route("/puppies/", methods = ['GET' , 'POST']) > def puppiesFunction(): > if request.method == 'GET': > return getAllPuppies() > As suspected the error is correct. You do not have a request object anywhere in your code. A quick Google tells me that it is part of Flask. So you need to import request from your flask module: from flask import Flask, request,,# any other names you use hth -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] if request.method == 'GET': NameError: global name 'request' is not defined
On 17Jan2016 20:13, sudipto mannawrote: Here is the code snippet: File#FlaskTest2.py from flask import Flask You need to import "request" from flask as well: from flask import Flask, request For others on this list: the Flask framework presents the current web request as a thread local global called "request"; simplifies writing handlers as the request information is readily available without having to pass it through function calls. Cheers, Cameron Simpson ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] pandas data frame
Bachir Bachir wrote: [Bachir, please send your mails to the list, not to me. That way you increase the likelihood to get a good answer] >> On Sunday, January 17, 2016 4:20 PM, Peter Otten <__pete...@web.de> >> wrote: >> Bachir Bachir via Tutor wrote: >>> Hello EverybodyI need to sort a dataframe according to a specific column >>> the create new dataframes according to the sorted columns each new >>> created dataframe should contain the list(set('the sorted element)any >>> help please, i am new in python and pandas thanks muchBachir >> >> Like this? >> >> >>> df >> >> foo bar >> 0 3.3 b >> 1 2.2 b >> 2 7.5 a >> 3 1.1 c >> 4 4.7 a >> >> [5 rows x 2 columns] >> >> >>> df.groupby("bar").sum() >> >> foo >> bar >> a12.2 >> b5.5 >> c1.1 >> >> [3 rows x 1 columns] > Thanks Peter ,This way but i want keep all the table elements only > separate them by a single column I'm sorry, I don't understand what you mean by "separate them by a single column". Can you give an example? What exactly should the sorted version of >> >>> df >> >> foo bar >> 0 3.3 b >> 1 2.2 b >> 2 7.5 a >> 3 1.1 c >> 4 4.7 a >> >> [5 rows x 2 columns] look like? ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] if request.method == 'GET': NameError: global name 'request' is not defined
On Mon, Jan 18, 2016 at 7:59 AM, sudipto mannawrote: > Thanks All. > The import request was missing and the issue was resolved upon calling that > module and initializing it. Hi Sudipto Manna, One last thing before you go on. Do a small retrospective: now that you know why things weren't working, look back at the original error message again. ### File "/Users/sudiptomanna/fullstack/vagrant/PythonData/FlaskTest2.py", line 8, in puppiesFunction if request.method == 'GET': NameError: global name 'request' is not defined ### Does the message in the error message make more sense to you now? Best of wishes! ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Should a "data" directory have a __init__.py file?
On 18 January 2016 at 04:27, Ben Finneywrote: > > The module import system will only recognise a directory as a “package” > (Python's technical term for “a point in the import hierarchy which > contains other things to import”) if that directory contains a file > named ‘__init__.py’. If you do not need the directory to also be > recognised as a package, it should not have such a file. That was the case prior to 3.3. However (I can't immediately test this but) now a directory without an __init__.py can be an implicit namespace package if it contains importable modules: https://www.python.org/dev/peps/pep-0420/ -- Oscar ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] How (and whether I should) use pkgutil.get_data()?
On 17 January 2016 at 23:37, boB Steppwrote: > 1) Is using pkgutil.get_data() the way I should be reading my data files? Generally yes, although it may be unnecessary. The main purpose of pkgutil.get_data is to transparently handle the case where your packages are imported from a zip file. The initial motivation for this was setuptools eggs but there are other reasons you might want to import your code from a zip file. > 2) The book says that this method will return "...a byte string > containing the raw contents of the file." (From page 409) Can I do all > of the normal things on this returned object that I could do if I used > "with open..."? Is it iterable? [OK, I confess up front. I am being > lazy here. I *could* create a file and try this out to see what > happens, so I will understand if I get chided (or worse!).] Just try it out :) > 3) Should I be using relative imports in my projects? I don't use relative imports. The supposed advantage of relative imports is that you can easily move a package to a different location in the import hierarchy. For example if you have a package called stuff and the modules inside stuff use relative imports to access each other then you could move stuff inside another package called things and have it be a package called things.stuff without needing to change the import lines. Personally I think that fixing up a few import lines is no big deal. I'd rather choose exactly where things is going to go in the import hierarchy and be done with it. If I need to move it into another package then I'll have to change the import statements inside things but that's easy to do. So I prefer absolute imports since they're less ambiguous. -- Oscar ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] if request.method == 'GET': NameError: global name 'request' is not defined
Thanks All. The import request was missing and the issue was resolved upon calling that module and initializing it. Thanks for the guidance. Regards, Sudipto Manna On Mon, Jan 18, 2016 at 3:55 AM, Cameron Simpsonwrote: > On 17Jan2016 20:13, sudipto manna wrote: > >> Here is the code snippet: >> >> File#FlaskTest2.py >> >> from flask import Flask >> > > You need to import "request" from flask as well: > > from flask import Flask, request > > For others on this list: the Flask framework presents the current web > request as a thread local global called "request"; simplifies writing > handlers as the request information is readily available without having to > pass it through function calls. > > Cheers, > Cameron Simpson > ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
[Tutor] Password decoding (was Re: Fwd:)
On 18/01/16 13:06, Deepak Nn wrote: > Please provide a python program to run a program (.exe) and get Hash > *exactly* as : > > 160 106 182 190 228 64 68 207 248 109 67 88 41 .The username to be > used is admin > .The *password* is what to be found out .The hash provided is of the > correct password .Mostly the password will be *13 char long *and has a > small chance of being all alpha characters .*GOOGLE DOES NOT ALLOW TO SENT > .EXE FILE .* > > Provide the where is change i have to make in the program to run the .exe > file that i have in my computer . > > Since i can't sent the .exe file .i am sending the screen shorts of the > program .*The text file of the program is included and few screenshots of > the the the result of the program is also included* . The list server does not generally let attachments through. Also, we will not do your homework for you, although we will try to help you do it. Please show us what you have tried so far and any error messages (in full). Ask specific questions and you will get specific help. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Simultaneous read and write on file
On 18/01/16 16:01, Anshu Kumar wrote: > Hello Everyone, > > I try below code in python 2.7.10 to first create and write into a file and > then read and write that file but what i get is just a file with new > content. > > with open('test.txt', 'wb+') as f: > ... f.write('this is test file.') > ... f.write('ok!!!') > ... with open('test.txt', 'wb+') as f: > ... a_str = f.read() + 'read the file' > ... f.seek(0) > ... f.write(a_str) > ... You called f.seek(0) which puts the cursor right back to the start of the file. Try writing without using the f.seek(). But since you only want to append, it would be better and safer to use append mode instead. The + modes are deceptively appealing but they are full of dangers for precisely the reasons you have discovered(*). You very rarely need them and you are better opening/closing the file and using explicit modes to read/write. (*)Another problem, apart from the risks of overwriting your data, is that the + modes will lock the file even while you are just reading it, which might cause a problem with shared files) > I have read in documentation that wb+ mode is for writing and reading. Am i > using wrong mode, should i use rb+ ? No, you should probably be separately using 'r' to read and 'a' to append. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
[Tutor] Simultaneous read and write on file
Hello Everyone, I try below code in python 2.7.10 to first create and write into a file and then read and write that file but what i get is just a file with new content. >>> with open('test.txt', 'wb+') as f: ... f.write('this is test file.') ... f.write('ok!!!') ... >>> with open('test.txt', 'wb+') as f: ... a_str = f.read() + 'read the file' ... f.seek(0) ... f.write(a_str) ... I have read in documentation that wb+ mode is for writing and reading. Am i using wrong mode, should i use rb+ ? Thanks and regards, Anshu ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Fwd:
> Please provide a python program to run a program (.exe) and get Hash > *exactly* as : > > 160 106 182 190 228 64 68 207 248 109 67 88 41 .The username to be > used is admin > .The *password* is what to be found out .The hash provided is of the > correct password .Mostly the password will be *13 char long *and has a > small chance of being all alpha characters . Hi Deepak, This doesn't seem like a beginner-level question. If I had a guess, it sounds more like something out of a shady rent-a-coder kind of thing. Unfortunately, I don't think we can help with this. Even if we did have the technical expertise, I still don't think we should help on this in the first place. If I'm understanding the question correctly, you're asking for brute password breaking, which goes against most professional codes of conduct. Example: https://www.acm.org/about-acm/acm-code-of-ethics-and-professional-conduct. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] An Advice to a dedicated beginner
On 19/01/16 01:15, Michael Appiah Boachie wrote: > I am a programming beginner. ... > when i sat on my own with a couple of materials and videos, > I have become very good at the basics of python and > completing various beginner courses and projects with ease > but however I have run into some kind of “beginner wall” > where I don’t know where or what to take on next. The first thing to do is to find a concept or project that you want to build. We can't really help you with that, it has to come from you. But maybe there is a routine task you perform that you could automate? Say, renaming photos that you load from a camera? Or bulk conversion of music files between formats? Or you have a spreadsheet with some macros that you could convert to an application, maybe using a database? Or maybe it's a game that you like? Or you could generate reports from some data source? Maybe from a web site? Once you have a project, you need to decide how to build it, and what modules you will be using. That second action is where my last book comes in. Python Projects is targeted at people like you, who know the basic language, but now want to do something with it. It covers: interacting with the OS, managing data, building UIs and web sites and using the Python test tools. I try to avoid too much self promotion on this list but it does sound like you might benefit from this one. What it doesn't cover much is how to design a program, that's a much more abstract concept and there are whole books on that topic. But, for a beginners project, it shouldn't be a huge issue, the important thing is to pick something and make a start. If you get stuck come back here. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
[Tutor] An Advice to a dedicated beginner
Hello Tutors, I am a programming beginner. Throughout college, I have not properly grasp anything the professor’s teach but when i sat on my own with a couple of materials and videos, I have become very good at the basics of python and completing various beginner courses and projects with ease but however I have run into some kind of “beginner wall” where I don’t know where or what to take on next. This is killing my excitement. I think this isn’t something new to experienced programmers to hear. That’s why I am asking for help. Please any advice would help a dedicated one here. I don’t know what to do next with the knowledge I have acquired. People keep saying “get into open source” , “do that and that”. I wish they actually knew how someone like me feel. There are so many videos, articles and materials to get you to know basics and also become a top expert but almost nothing on how to transition into that. That’s exactly how I’m feeling. Hoping to receive some kind words. Michael ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] An Advice to a dedicated beginner
Alan Gauldwrites: > On 19/01/16 01:15, Michael Appiah Boachie wrote: > > I am a programming beginner. ... > > when i sat on my own with a couple of materials and videos, > > I have become very good at the basics of python and > > completing various beginner courses and projects with ease > > but however I have run into some kind of “beginner wall” > > where I don’t know where or what to take on next. > > The first thing to do is to find a concept or project > that you want to build. We can't really help you with > that, it has to come from you. But maybe there is a > routine task you perform that you could automate? That's a worthwhile recommendation. I usually recommend something different: Pick an *existing* code base that you're already using – Python is used in a great many tools that you probably rely on, so this shouldn't be difficult – and get the source code. Find small improvements to make, discuss them with peers and on this forum, and with the developers of that code base. Making small improvements to something you already use will avoid the “blank slate” paralysis from having too many options be motivational, while also being very motivational: you already can think of improvements you would like to make, and getting them implemented will be very satisfying. It also focusses on an often-dismissed aspect of learning to program: you need to program *with other people*, and your code is a means of communicating with those people. Learning how to do that, and learning (by doing) that there is no shame in starting small and unskilled, is a necessary part of the craft. -- \ “The long-term solution to mountains of waste is not more | `\ landfill sites but fewer shopping centres.” —Clive Hamilton, | _o__)_Affluenza_, 2005 | Ben Finney ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Simultaneous read and write on file
Hello, >> I have read in documentation that wb+ mode is for writing and >> reading. Am i using wrong mode, should i use rb+ ? > >Use w+ to create a new file, opened with read and write access. Use >r+ to open an existing file with read and write access. Unlike w+, >r+ does not truncate the file and will not create a new file. [snip] >FYI, in terms of POSIX open flags [1], the file mode gets mapped as follows: > > | flags | no + | + >- >r | | O_RDONLY | O_RDWR >w | O_CREAT, O_TRUNC | O_WRONLY | O_RDWR >a | O_CREAT, O_APPEND | O_WRONLY | O_RDWR > >Python 3 only: > >x | O_CREAT, O_EXCL | O_WRONLY | O_RDWR > >In terms of Windows open dispositions and access modes [2], the file >mode gets mapped as follows: > > | disposition | no + |+ >--- >r | OPEN_EXISTING | GENERIC_READ | GENERIC_READ, GENERIC_WRITE >w | CREATE_ALWAYS | GENERIC_WRITE | GENERIC_READ, GENERIC_WRITE >a | OPEN_ALWAYS | GENERIC_WRITE | GENERIC_READ, GENERIC_WRITE > >Python 3 only: > >x | CREATE_NEW| GENERIC_WRITE | GENERIC_READ, GENERIC_WRITE The above is a very handy chart. Did you find this somewhere, eryk sun, or is this from your own knowledge and experience? This might benefit an experienced Windows/POSIX user trying to understand open() in Python if it were available in the standard documentation. Thanks for mapping this to common operating systems. I had inferred this already, but this is a great summary. -Martin -- Martin A. Brown http://linux-ip.net/ ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Simultaneous read and write on file
On 18Jan2016 20:41, Martin A. Brownwrote: Yes and so have I. Maybe twice in 30 years of programming. [...] I may have done it a little more than that; I agree it is very rare. I may be biased because I was debugging exactly this last week. (Which itself is an argument against mixed rerad/write with one file - it was all my own code and I spent over a day chasing this because I was looking in the wrong spot). Oh yes. Ooof. Today's decisions are tomorrow's albatross. Actually I have good reason to mix these in this instance, and now that it is debugged it is reliable and more efficient to boot. [...] Tip for new players: if you do any .write()s, remember to do a .flush() before doing a seek or a read That's exactly my point. There are so many things you have to do extra when working in mixed mode. Too easy to treat things like normal mode files and get it wrong. Experts can do it and make it work, but mostly it's just not needed. Yes. You're write - for simplicity and reliability two distinct open file instances are much easier. Yes, he's write [sic]. He writes a bunch! ;) Alas, I have a tendency to substitute homophones, or near homophones, when typing in a hurry. You'll see this in a bunch of my messages. More annoyingly, some are only visible when I reread a posted message instead of when I was proofreading prior to send. [Homonyms mess me up when I'm typing, all sew.] Homonyms too. Cheers, Cameron Simpson ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Simultaneous read and write on file
On 18Jan2016 21:07, ALAN GAULDwrote: On 18/01/16 20:43, Cameron Simpson wrote: The + modes are deceptively appealing but they are full of dangers for precisely the reasons you have discovered(*). You very rarely need them and you are better opening/closing the file and using explicit modes to read/write. But if he wants to mix the modes, he certainly should be experimenting. Having a file open for read and write is sometimes useful; I do it myself in certain circumstances. Yes and so have I. Maybe twice in 30 years of programming. It's sometimes necessary but it's much, much harder to get right and very easy to get wrong, usually with data corruption as a result. I may have done it a little more than that; I agree it is very rare. I may be biased because I was debugging exactly this last week. (Which itself is an argument against mixed rerad/write with one file - it was all my own code and I spent over a day chasing this because I was looking in the wrong spot). So for a beginner I would never encourage it. For an experienced programmer sure' if there is no other option (and especially if you have fixed size records where things get easier). Tip for new players: if you do any .write()s, remember to do a .flush() before doing a seek or a read That's exactly my point. There are so many things you have to do extra when working in mixed mode. Too easy to treat things like normal mode files and get it wrong. Experts can do it and make it work, but mostly it's just not needed. Yes. You're write - for simplicity and reliability two distinct open file instances are much easier. Disagree. As far as his question goes, "wb+" is a correct mode for what he is trying. Whether it is a sensible approach depends very much on what he is doing with his files. I'm not sure we know what he(?) is trying. We only know he successfully overwrote his data and that apparently was not his intention. There are use cases where it makes sense but in most cases you can get by just fine without. Yeah. I think I was more narked by your not answering his "is wb+ correct" literally; it may be only rarely the reasonable course, but for what he was actually _asking_ wb+ is correct. He may not be doing the best design but as you say we don't know his actual use case. For the narkiness I apologise. Cheers, Cameron Simpson ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Simultaneous read and write on file
On 18Jan2016 22:29, Peter Otten <__pete...@web.de> wrote: Anshu Kumar wrote: I have read in documentation that wb+ mode is for writing and reading. Am i using wrong mode, should i use rb+ ? Quoting https://docs.python.org/2.7/library/functions.html#open """ note that 'w+' truncates the file. """ That's why you lose the file's current content, and, yes, "r+b" would avoid that. And I stand corrected; I should have paid more attention. Thanks! Cheers, Cameron Simpson___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Simultaneous read and write on file
Hi all, The + modes are deceptively appealing but they are full of dangers for precisely the reasons you have discovered(*). > >> Yes and so have I. Maybe twice in 30 years of programming. It's >> sometimes necessary but it's much, much harder to get right and >> very easy to get wrong, usually with data corruption as a result. Yes. Yes. Yes. Yes. Yes. > I may have done it a little more than that; I agree it is very > rare. I may be biased because I was debugging exactly this last > week. (Which itself is an argument against mixed rerad/write with > one file - it was all my own code and I spent over a day chasing > this because I was looking in the wrong spot). Oh yes. Ooof. Today's decisions are tomorrow's albatross. Speaking of which, I have an albatross in very good condition and I'm looking for a buyer. Anybody interested in a short-tailed albatross with good manners. Looking for a good home. >> So for a beginner I would never encourage it. For an experienced >> programmer sure' if there is no other option (and especially if >> you have fixed size records where things get easier). >> >>> Tip for new players: if you do any .write()s, remember to do a >>> .flush() before doing a seek or a read >> >> That's exactly my point. There are so many things you have to do >> extra when working in mixed mode. Too easy to treat things like >> normal mode files and get it wrong. Experts can do it and make it >> work, but mostly it's just not needed. > > Yes. You're write - for simplicity and reliability two distinct > open file instances are much easier. Yes, he's write [sic]. He writes a bunch! ;) [Homonyms mess me up when I'm typing, all sew.] OK, a bit more seriously, I will add a thought or two. Modern filesystems are beautiful. They are fast, reliable and efficient. Application software, e.g. Python, can be hairy (see the points of Alan and Cameron earlier in this thread). Why not take advantage of filesystem atomicity, a feature guarantee to userspace from all (?) modern local filesystems. Options: * If disk throughput is not a problem, then there's practically nothing but a benefit to reading from input file, writing to output file, closing both and renaming (effectively squashing the original file) import os A = open('a', 'w') A.write('hammy') A.close() A = open('a', 'r') B = open('b', 'w') data = A.read() # -- processing handled data = data.replace('m', 'p') #here, until happy B.write(data) B.close() A.close() os.rename(B.name, A.name) # -- atomic [0] * Alternative: If disk throughput is a problem, this is an argument for using a database system where this class of data integrity problem has been solved for the application developer. I'd suggest measuring the amount of time it takes to read, rewrite and os.rename() the entire file before deciding you need to undertake the massive complexity of modifying an existing file in situ. If you can avoid modifying an existing file, don't bother with it. You will likely bring yourself (and maybe even others) headache. For example, if another reader of the file comes along while you are performing your in situ modification magic tricks, they (and you) will have no guarantees about what data they will receive. That will be left up to the operating system (i.e. kernel). So, take control of the data back into your own hands by taking adavantage of the beauty of the filesystem. Filesystem atomicity! Good luck, -Martin [0] Or just about as close as conceivably possible to atomic as you can be guaranteed in userspace applications. -- Martin A. Brown http://linux-ip.net/ ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
[Tutor] Beautiful Soup
Hi Python Tutors, I am currently able to strip down to the string I want. However, I have problems with the JSON script and I am not sure how to slice it into a dictionary. import urllib import json import requests from bs4 import BeautifulSoup url = 'https://bochk.etnet.com.hk/content/bochkweb/eng/quote_transaction_daily_history.php?code=6881\ =F=09=16=S=44c99b61679e019666f0570db51ad932=0=0' def web_scraper(url): response = requests.get(url) html = response.content soup = BeautifulSoup(html, 'lxml') stock1 = soup.findAll('script')[4].string stock2 = stock1.split() stock3 = stock2[3] # is stock3 sufficient to process as JSON or need further cleaning?? text = json.dumps(stock3) print(text) web_scraper(url) If it is possible, please give me some pointers. Thank you Regards, Henry ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Simultaneous read and write on file
On Mon, Jan 18, 2016 at 10:04 PM, Martin A. Brownwrote: > The above is a very handy chart. Did you find this somewhere, eryk > sun, or is this from your own knowledge and experience? The mapping to POSIX open flags is from a table in the POSIX fopen spec [1] as well as the Windows CRT docs for fopen (see the equivalent oflag table) [2]. The flags for "x" mode can be verified in the Python 3 source [3]. The "x" mode flags are also mentioned in GNU's libc docs for fopen [4] and open-time flags [5]. The mapping from POSIX open flags to Windows create disposition and access modes is from the CRT source for the _open function. I can't provide a source link, but the source code is distributed with MSVC. For VS2015 it's in lowio/open.cpp, decode_access_flags() and decode_open_create_flags(). [1]: http://pubs.opengroup.org/onlinepubs/9699919799/functions/fopen.html [2]: https://msdn.microsoft.com/en-us/library/yeby3zcb [3]: https://hg.python.org/cpython/file/v3.5.1/Modules/_io/fileio.c#l302 [4]: https://www.gnu.org/software/libc/manual/html_node/Opening-Streams.html [5]: https://www.gnu.org/software/libc/manual/html_node/Open_002dtime-Flags.html ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Simultaneous read and write on file
Anshu Kumar wrote: > Hello Everyone, > > I try below code in python 2.7.10 to first create and write into a file > and > then read and write that file but what i get is just a file with new > content. > > with open('test.txt', 'wb+') as f: > ... f.write('this is test file.') > ... f.write('ok!!!') > ... with open('test.txt', 'wb+') as f: > ... a_str = f.read() + 'read the file' > ... f.seek(0) > ... f.write(a_str) > ... > > > I have read in documentation that wb+ mode is for writing and reading. Am > i using wrong mode, should i use rb+ ? Quoting https://docs.python.org/2.7/library/functions.html#open """ note that 'w+' truncates the file. """ That's why you lose the file's current content, and, yes, "r+b" would avoid that. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Simultaneous read and write on file
On 18/01/16 20:43, Cameron Simpson wrote: >> The + modes are deceptively appealing but they are full of dangers >> for precisely the reasons you have discovered(*). You very rarely >> need them and you are better opening/closing the file and >> using explicit modes to read/write. > > But if he wants to mix the modes, he certainly should be experimenting. > Having > a file open for read and write is sometimes useful; I do it myself in certain > circumstances. Yes and so have I. Maybe twice in 30 years of programming. It's sometimes necessary but it's much, much harder to get right and very easy to get wrong, usually with data corruption as a result. So for a beginner I would never encourage it. For an experienced programmer sure' if there is no other option (and especially if you have fixed size records where things get easier). > Tip for new players: if you do any .write()s, remember to do a .flush() > before > doing a seek or a read That's exactly my point. There are so many things you have to do extra when working in mixed mode. Too easy to treat things like normal mode files and get it wrong. Experts can do it and make it work, but mostly it's just not needed. > Disagree. As far as his question goes, "wb+" is a correct mode for what he is > trying. Whether it is a sensible approach depends very much on what he is > doing > with his files. I'm not sure we know what he(?) is trying. We only know he successfully overwrote his data and that apparently was not his intention. There are use cases where it makes sense but in most cases you can get by just fine without. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] pandas data frame
Bachir Bachir via Tutor wrote: > Hi Peter Thank you much for your help its very appreciated ,bellow is an > example of what i need. The main dataframe ( need to be separated into > separate dataframes ). The desired dataframes output correctly as i need > it to be,Thanks much Main dataframendx V_idAverageMean > Peak 0 1 3 2 51 > 2 2 1 62 3 > 4 1 83 1 2 > 2 74 2 3 3 > 65 3 5 3 46 >1 1 1 87 2 > 2 5 108 3 5 > 5 99 1 2 5 > 10102 5 5 911 > 3 4 3 10 Dataframe-1ndx V_id > Average MeanPeak 0 1 3 2 > 53 1 2 2 76 1 1 > 1 89 1 2 5 10 > Dataframe-2ndx V_idAveragMean Peak 1 2 > 2 1 64 2 3 3 > 67 2 2 5 10102 > 5 5 9 Dataframe-3ndx V_idAverageMean > Peak 2 3 4 1 85 3 > 5 3 48 3 5 5 > 9113 4 3 10 Bachir I see that you got an answer over at comp.python.pydata where you managed to get the format a bit more readable ;) I recommend that you go with Goyo's answer, but instead of the hacky use of locals() > >>> locals().update((('dataframe_{}'.format(k), grouper.get_group(k)) > ... for k in grouper.groups)) I recommend that you put the groups into a normal dict groups = {k: g.get_group(k) for k in g.groups} and then access the partial data frames with v_id = 2 groups[v_id] ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Simultaneous read and write on file
On 18Jan2016 16:29, ALAN GAULDwrote: On 18/01/16 16:01, Anshu Kumar wrote: I try below code in python 2.7.10 to first create and write into a file and then read and write that file but what i get is just a file with new content. with open('test.txt', 'wb+') as f: ... f.write('this is test file.') ... f.write('ok!!!') ... with open('test.txt', 'wb+') as f: ... a_str = f.read() + 'read the file' ... f.seek(0) ... f.write(a_str) ... You called f.seek(0) which puts the cursor right back to the start of the file. Try writing without using the f.seek(). Agreed. But since you only want to append, it would be better and safer to use append mode instead. The + modes are deceptively appealing but they are full of dangers for precisely the reasons you have discovered(*). You very rarely need them and you are better opening/closing the file and using explicit modes to read/write. But if he wants to mix the modes, he certainly should be experimenting. Having a file open for read and write is sometimes useful; I do it myself in certain circumstances. Tip for new players: if you do any .write()s, remember to do a .flush() before doing a seek or a read - unlike the C stdio library where the flush is automatic in Python the io classes require you to flush written data if you read or seek. (You don't have to flush before close, close does that for you.) (*)Another problem, apart from the risks of overwriting your data, is that the + modes will lock the file even while you are just reading it, which might cause a problem with shared files) Only on Windows. On UNIX everything is fine unless you go out of your way to make things harder with locking. I have read in documentation that wb+ mode is for writing and reading. Am i using wrong mode, should i use rb+ ? No, you should probably be separately using 'r' to read and 'a' to append. Disagree. As far as his question goes, "wb+" is a correct mode for what he is trying. Whether it is a sensible approach depends very much on what he is doing with his files. He _may_ be safer opening the file twice - once for "rb" and once for "ab" - because he does not have to juggle the modes, but it isn't necessarily what he wants. Cheers, Cameron Simpson ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Simultaneous read and write on file
On Mon, Jan 18, 2016 at 10:01 AM, Anshu Kumarwrote: > I have read in documentation that wb+ mode is for writing and reading. Am i > using wrong mode, should i use rb+ ? Use w+ to create a new file, opened with read and write access. Use r+ to open an existing file with read and write access. Unlike w+, r+ does not truncate the file and will not create a new file. To clarify what Alan said, the "+" modes do not 'lock' the file. On Windows you may experience a sharing violation if another program doesn't share write access. This isn't quite the same as a file lock. All open references to a file have to agree on read, write, and delete sharing. For example, when Notepad saves a file, it tries to open a handle that shares only read access. If this open succeeds, Notepad knows that it's the only writer. (It doesn't care if there are multiple readers.) On the other hand, if some other program has the file open with write or delete access, Notepad's open will fail with a sharing violation. Then it displays a message box saying it can't save the file because another process has it open, and it offers to save the file using a different name. FYI, in terms of POSIX open flags [1], the file mode gets mapped as follows: | flags | no + | + - r | | O_RDONLY | O_RDWR w | O_CREAT, O_TRUNC | O_WRONLY | O_RDWR a | O_CREAT, O_APPEND | O_WRONLY | O_RDWR Python 3 only: x | O_CREAT, O_EXCL | O_WRONLY | O_RDWR In terms of Windows open dispositions and access modes [2], the file mode gets mapped as follows: | disposition | no + |+ --- r | OPEN_EXISTING | GENERIC_READ | GENERIC_READ, GENERIC_WRITE w | CREATE_ALWAYS | GENERIC_WRITE | GENERIC_READ, GENERIC_WRITE a | OPEN_ALWAYS | GENERIC_WRITE | GENERIC_READ, GENERIC_WRITE Python 3 only: x | CREATE_NEW| GENERIC_WRITE | GENERIC_READ, GENERIC_WRITE O_APPEND entails an implicit seek to the end of a file before every write. Note that Python uses the Windows CRT for file access, which doesn't use the native FILE_APPEND_DATA access to implement O_APPEND. Instead it opens the file for generic writing and manually updates the file pointer. This may cause problems if the security descriptor only allows appending to a file. In this case you'd have to call CreateFile directly (e.g. via ctypes or win32api). [1]: http://pubs.opengroup.org/onlinepubs/9699919799/functions/open.html [2]: https://msdn.microsoft.com/en-us/library/aa363858 ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor