Re: Adding 'download' column to existing 'visitors' table (as requested)
On 2013-11-09 04:27, Piet van Oostrum wrote: Sibylle Koczian writes: Am 07.11.2013 14:14, schrieb Piet van Oostrum: Nick the Gr33k writes: I have decided to take your advice. I wasn't able to fit those 'lists' of mine into MySQL's varchar() datatype after converting them to long strings and that sads me. My implementation is like the following. I do not use an extra table of downlaods that i asoociate with table visitors with a foreing key but decided to add an additional 'download' column into the existant visitors table: Nikos, you are an excellent member of the Greek society. Listening to you makes it so much easier to understand the problems that your country has. Is there any reason at all to insult all other Greek readers of this newsgroup? I was talking about the Greek nation. That doesn't imply that every single Greek is like that. Just a majority of Greeks? How comforting. Please don't. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- https://mail.python.org/mailman/listinfo/python-list
Re: Adding 'download' column to existing 'visitors' table (as requested)
Sibylle Koczian writes: > Am 07.11.2013 14:14, schrieb Piet van Oostrum: >> Nick the Gr33k writes: >> >>> I have decided to take your advice. >>> I wasn't able to fit those 'lists' of mine into MySQL's varchar() >>> datatype after converting them to long strings and that sads me. >>> >>> My implementation is like the following. >>> I do not use an extra table of downlaods that i asoociate with table >>> visitors with a foreing key but decided to add an additional 'download' >>> column into the existant visitors table: >> >> Nikos, you are an excellent member of the Greek society. Listening to you >> makes it so much easier to understand the problems that your country has. > > Is there any reason at all to insult all other Greek readers of this > newsgroup? I was talking about the Greek nation. That doesn't imply that every single Greek is like that. -- Piet van Oostrum WWW: http://pietvanoostrum.com/ PGP key: [8DAE142BE17999C4] -- https://mail.python.org/mailman/listinfo/python-list
Re: Adding 'download' column to existing 'visitors' table (as requested)
Στις 7/11/2013 8:08 μμ, ο/η Neil Cerutti έγραψε: On 2013-11-07, ?? ?? wrote: I called you an idiot, because in your previous and current message you called me too. I know that splitting information across tables and maintain foreign keys for retain relationships between them is a necessary thing but in my case i only just an extra pieces of information to eb associated with my visitor, a possible file download. and i have decided just to add an extra colum to the existing 'visitors' database and this is adequate. Non-normalized data is sometimes a fine idea. How you plan to use the data once it is stored will be the deciding factor. Exactly. One big win with databases is that you can query them really easily using SQL. Non-normalized data negates that advantage. How would you write a query to discover all the visitors who downloaded file XYZ? With your storage scheme, you can't. So by storing the data this way, you are promising yourself that you'll never need to write that query, or at least, you won't need to do it very often. That would be a problem yes. But as you said above the deciding factor is the "how" we plan to use out stored data. And my plan is to just display the records of all visitors per webpage with the last column being a list of this specific visitors 'downloads' as can be seen visually here: http://superhost.gr/?show=log&page=index.html 'Δεν πραγματοποίηθηκαν ακόμη!' mean that this visitor hasn't download anything yet, if he does a drop down menu will appear in that place displaying his file picks. People can download files from here:'http://superhost.gr/?page=files.py (these torrent are just for testing reasons. later i will put my own selection of files) -- https://mail.python.org/mailman/listinfo/python-list
Re: Adding 'download' column to existing 'visitors' table (as requested)
On 2013-11-07, ?? ?? wrote: > I called you an idiot, because in your previous and current > message you called me too. > > I know that splitting information across tables and maintain > foreign keys for retain relationships between them is a > necessary thing but in my case i only just an extra pieces of > information to eb associated with my visitor, a possible file > download. and i have decided just to add an extra colum to the > existing 'visitors' database and this is adequate. Non-normalized data is sometimes a fine idea. How you plan to use the data once it is stored will be the deciding factor. One big win with databases is that you can query them really easily using SQL. Non-normalized data negates that advantage. How would you write a query to discover all the visitors who downloaded file XYZ? With your storage scheme, you can't. So by storing the data this way, you are promising yourself that you'll never need to write that query, or at least, you won't need to do it very often. > I still don't know why you push me to create an extra table > instead. Because it's usually the right thing to do. -- Neil Cerutti -- https://mail.python.org/mailman/listinfo/python-list
Re: Adding 'download' column to existing 'visitors' table (as requested)
Στις 7/11/2013 5:11 μμ, ο/η Joel Goldstick έγραψε: On Thu, Nov 7, 2013 at 9:37 AM, Chris Angelico wrote: On Fri, Nov 8, 2013 at 1:09 AM, Νίκος Αλεξόπουλος wrote: Why create a whole new 'downloads' table and associate it with the with a foreign key with the 'visitors' table you idiot when you can just have an extra column at the end of the current 'visitor's table? Both 'downloader' and 'downlaod' is associated by being in the same record. By your logic every time we want to store an extra piece of information we have to create an extra database table. Too much hussle for no good reason First of all Nikos, you are not qualified to determine if there is reason to design a database in any particular way, since you don't understand what a relational database is. Secondly, in earlier threads I provided you with a link to a wikipedia article about first normal form and why it is a necessary component of data base design. If you don't want to use a database, good for you, but if you can't get your website to work and you ask for help, then respond that the help is 'too much hussle', you are disrespectful. You don't bother me because you are lazy, and arogant, and whiny. Or that you lack skills. You are a cargo cult programmer. I know you won't look that up. You bother me because you are disrespectful. The most disrespectful person I have ever encountered on line. On a side note to whoever hacks into Nick the idiots website -- why not just take it down. Then he won't have any need to spend his 20 euros, and there will be no code to fix. Problem solved! I called you an idiot, because in your previous and current message you called me too. I know that splitting information across tables and maintain foreign keys for retain relationships between them is a necessary thing but in my case i only just an extra pieces of information to eb associated with my visitor, a possible file download. and i have decided just to add an extra colum to the existing 'visitors' database and this is adequate. I still don't know why you push me to create an extra table instead. It may seem that i'm clue resistant sometimes and i'm but this is not because out of arogance but as a result of failign to under -- https://mail.python.org/mailman/listinfo/python-list
Re: Adding 'download' column to existing 'visitors' table (as requested)
On 2013-11-07 17:03, Sibylle Koczian wrote: > > Nikos, you are an excellent member of the Greek society. > > Listening to you makes it so much easier to understand the > > problems that your country has. > > Is there any reason at all to insult all other Greek readers of > this newsgroup? Greece is no more represented by Nikos than any other nations are represented by their ignorant. When I start to feel ill-will towards Greece because of Nikos, I also have to remember that the country has also produced great technologists like Lea Verou and classical art & philosophy. Then I just wonder why Nikos doesn't take advantage of the resources his home country provides. :-/ -tkc -- https://mail.python.org/mailman/listinfo/python-list
Re: Adding 'download' column to existing 'visitors' table (as requested)
Am 07.11.2013 14:14, schrieb Piet van Oostrum: Nick the Gr33k writes: I have decided to take your advice. I wasn't able to fit those 'lists' of mine into MySQL's varchar() datatype after converting them to long strings and that sads me. My implementation is like the following. I do not use an extra table of downlaods that i asoociate with table visitors with a foreing key but decided to add an additional 'download' column into the existant visitors table: Nikos, you are an excellent member of the Greek society. Listening to you makes it so much easier to understand the problems that your country has. Is there any reason at all to insult all other Greek readers of this newsgroup? -- https://mail.python.org/mailman/listinfo/python-list
Re: Adding 'download' column to existing 'visitors' table (as requested)
On 2013-11-07, ?? ?? wrote: > [nothing my newsreader cared to keep] OK, so when posting a follow-up, Nikos is now putting his entire posting into his signature? This guy's a hoot-and-a-half! -- Grant Edwards grant.b.edwardsYow! Somewhere in Tenafly, at New Jersey, a chiropractor gmail.comis viewing "Leave it to Beaver"! -- https://mail.python.org/mailman/listinfo/python-list
Re: Adding 'download' column to existing 'visitors' table (as requested)
On Thu, Nov 7, 2013 at 9:37 AM, Chris Angelico wrote: > On Fri, Nov 8, 2013 at 1:09 AM, Νίκος Αλεξόπουλος > wrote: >> Why create a whole new 'downloads' table and associate it with the with a >> foreign key with the 'visitors' table you idiot when you can just have an >> extra column at the end of the current 'visitor's table? >> >> Both 'downloader' and 'downlaod' is associated by being in the same record. >> >> By your logic every time we want to store an extra piece of information we >> have to create an extra database table. >> >> Too much hussle for no good reason First of all Nikos, you are not qualified to determine if there is reason to design a database in any particular way, since you don't understand what a relational database is. Secondly, in earlier threads I provided you with a link to a wikipedia article about first normal form and why it is a necessary component of data base design. If you don't want to use a database, good for you, but if you can't get your website to work and you ask for help, then respond that the help is 'too much hussle', you are disrespectful. You don't bother me because you are lazy, and arogant, and whiny. Or that you lack skills. You are a cargo cult programmer. I know you won't look that up. You bother me because you are disrespectful. The most disrespectful person I have ever encountered on line. On a side note to whoever hacks into Nick the idiots website -- why not just take it down. Then he won't have any need to spend his 20 euros, and there will be no code to fix. Problem solved! -- Joel Goldstick http://joelgoldstick.com -- https://mail.python.org/mailman/listinfo/python-list
Re: Adding 'download' column to existing 'visitors' table (as requested)
On Fri, Nov 8, 2013 at 1:09 AM, Νίκος Αλεξόπουλος wrote: > Why create a whole new 'downloads' table and associate it with the with a > foreign key with the 'visitors' table you idiot when you can just have an > extra column at the end of the current 'visitor's table? > > Both 'downloader' and 'downlaod' is associated by being in the same record. > > By your logic every time we want to store an extra piece of information we > have to create an extra database table. > > Too much hussle for no good reason Go to your local library and pick up a book on database design - or possibly you'll find it on Wikipedia. There ARE good reasons for the "hassle" of normalization. There are times when you consciously denormalize (I often read tables into memory for a (read-only) cache, and denormalize aggressively), but the rule of thumb is: It's normal to normalize. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Adding 'download' column to existing 'visitors' table (as requested)
Στις 7/11/2013 3:52 μμ, ο/η Joel Goldstick έγραψε: 2. the download is remained associated with the person that made the download since all this info is placed in the same record. just think, all those folks who figured out databases were wrong. Nikos has shown that you just need to put everything in a single record. Wow! look at that record with everything in it! Its so cool and it helped to refrain Nikos the idiot! -- Why create a whole new 'downloads' table and associate it with the with a foreign key with the 'visitors' table you idiot when you can just have an extra column at the end of the current 'visitor's table? Both 'downloader' and 'downlaod' is associated by being in the same record. By your logic every time we want to store an extra piece of information we have to create an extra database table. Too much hussle for no good reason -- https://mail.python.org/mailman/listinfo/python-list
Re: Adding 'download' column to existing 'visitors' table (as requested)
On Thu, Nov 7, 2013 at 8:37 AM, Νίκος Αλεξόπουλος wrote: > Στις 6/11/2013 7:59 μμ, ο/η Denis McMahon έγραψε: > >> On Wed, 06 Nov 2013 09:30:03 +0200, Nick the Gr33k wrote: >> >>> I have decided to take your advice. >> >> >> No you haven't. You only think you have, but really you either haven't >> understood the advice at all. >> >>> My implementation is like the following. >>> I do not use an extra table of downlaods that i asoociate with table >>> visitors with a foreing key but decided to add an additional 'download' >>> column into the existant visitors table: >> >> >> No no no no no no no no no no nononononono no! >> >> That's *NOT* the right way to do it. >> >> And this is where I finally and terminally give up trying to help you. >> I've had enough. You refuse to learn the right way to do it. You won't >> listen to the opinions and suggestions of people with a great deal more >> experience than you have in such matters. It's not going to work properly >> in the end. I refuse to be associated with it any further. >> > > > -- > Denis, you may choose to not help any further, thats acceptable as you > personal choice. > > I have to inform you though that my solution of adding an extra 'download' > column in my 'visitors' table has the benefits of > > 1. refrain me for creating one more table refraining you is a very good thing > 2. the download is remained associated with the person that made the > download since all this info is placed in the same record. just think, all those folks who figured out databases were wrong. Nikos has shown that you just need to put everything in a single record. Wow! look at that record with everything in it! Its so cool and it helped to refrain Nikos the idiot! > > My solution works just fine and is giving no problems. Great, now that you have not problems, you might consider going away forever so as not to cause other people problems! > I cant overcome the urge though to try to use some database that can hold > lists to a single. You need to see a therapist to help with overcoming your urges. We are not qualified in that area. > -- > https://mail.python.org/mailman/listinfo/python-list -- Joel Goldstick http://joelgoldstick.com -- https://mail.python.org/mailman/listinfo/python-list
Re: Adding 'download' column to existing 'visitors' table (as requested)
Στις 6/11/2013 7:59 μμ, ο/η Denis McMahon έγραψε: On Wed, 06 Nov 2013 09:30:03 +0200, Nick the Gr33k wrote: I have decided to take your advice. No you haven't. You only think you have, but really you either haven't understood the advice at all. My implementation is like the following. I do not use an extra table of downlaods that i asoociate with table visitors with a foreing key but decided to add an additional 'download' column into the existant visitors table: No no no no no no no no no no nononononono no! That's *NOT* the right way to do it. And this is where I finally and terminally give up trying to help you. I've had enough. You refuse to learn the right way to do it. You won't listen to the opinions and suggestions of people with a great deal more experience than you have in such matters. It's not going to work properly in the end. I refuse to be associated with it any further. -- Denis, you may choose to not help any further, thats acceptable as you personal choice. I have to inform you though that my solution of adding an extra 'download' column in my 'visitors' table has the benefits of 1. refrain me for creating one more table 2. the download is remained associated with the person that made the download since all this info is placed in the same record. My solution works just fine and is giving no problems. I cant overcome the urge though to try to use some database that can hold lists to a single -- https://mail.python.org/mailman/listinfo/python-list
Re: Adding 'download' column to existing 'visitors' table (as requested)
Nick the Gr33k writes: > I have decided to take your advice. > I wasn't able to fit those 'lists' of mine into MySQL's varchar() > datatype after converting them to long strings and that sads me. > > My implementation is like the following. > I do not use an extra table of downlaods that i asoociate with table > visitors with a foreing key but decided to add an additional 'download' > column into the existant visitors table: Nikos, you are an excellent member of the Greek society. Listening to you makes it so much easier to understand the problems that your country has. -- Piet van Oostrum WWW: http://pietvanoostrum.com/ PGP key: [8DAE142BE17999C4] -- https://mail.python.org/mailman/listinfo/python-list
Re: Adding 'download' column to existing 'visitors' table (as requested)
On 06/11/2013 17:59, Denis McMahon wrote: On Wed, 06 Nov 2013 09:30:03 +0200, Nick the Gr33k wrote: I have decided to take your advice. No you haven't. You only think you have, but really you either haven't understood the advice at all. My implementation is like the following. I do not use an extra table of downlaods that i asoociate with table visitors with a foreing key but decided to add an additional 'download' column into the existant visitors table: No no no no no no no no no no nononononono no! That's *NOT* the right way to do it. And this is where I finally and terminally give up trying to help you. I've had enough. You refuse to learn the right way to do it. You won't listen to the opinions and suggestions of people with a great deal more experience than you have in such matters. It's not going to work properly in the end. I refuse to be associated with it any further. Queen "Another one bites the dust". -- Python is the second best programming language in the world. But the best has yet to be invented. Christian Tismer Mark Lawrence -- https://mail.python.org/mailman/listinfo/python-list
Re: Adding 'download' column to existing 'visitors' table (as requested)
On Wednesday, November 6, 2013 11:29:11 PM UTC+5:30, Denis McMahon wrote: > On Wed, 06 Nov 2013 09:30:03 +0200, Nick the Gr33k wrote: > > I have decided to take your advice. > No you haven't. You only think you have, but really you either haven't… No, you think that he thinks that he has. Of course more correctly, I think that you think that he thinks that he has whereas in fact (I think) that he doesn't think that he has. [Sorry couldn't resist!] -- https://mail.python.org/mailman/listinfo/python-list
Re: Adding 'download' column to existing 'visitors' table (as requested)
On Wed, 06 Nov 2013 09:30:03 +0200, Nick the Gr33k wrote: > I have decided to take your advice. No you haven't. You only think you have, but really you either haven't understood the advice at all. > My implementation is like the following. > I do not use an extra table of downlaods that i asoociate with table > visitors with a foreing key but decided to add an additional 'download' > column into the existant visitors table: No no no no no no no no no no nononononono no! That's *NOT* the right way to do it. And this is where I finally and terminally give up trying to help you. I've had enough. You refuse to learn the right way to do it. You won't listen to the opinions and suggestions of people with a great deal more experience than you have in such matters. It's not going to work properly in the end. I refuse to be associated with it any further. -- Denis McMahon, denismfmcma...@gmail.com -- https://mail.python.org/mailman/listinfo/python-list
Re: Adding 'download' column to existing 'visitors' table (as requested)
Στις 6/11/2013 9:38 πμ, ο/η Nick the Gr33k έγραψε: Ah great!!! I just examined my other MySQL database which just stored webpages and their corresponding visits and voila. Someone was able to pass values into my counters table: look: http://superhost.gr/?show=stats thats why it didn't had 1 or 2 or 3 as 'counterID' but more values were present. Someone successfully manipulated this part of my code: if cookieID != 'nikos' and ( os.path.exists( path + page ) or os.path.exists( cgi_path + page ) ) and re.search( r'(amazon|google|proxy|cloud|reverse|fetch|msn|who|spider|crawl|ping)', host ) is None: try: # if first time for webpage; create new record( primary key is automatic, hit is defaulted ), if page exists then update record cur.execute('''INSERT INTO counters (url) VALUES (%s) ON DUPLICATE KEY UPDATE hits = hits + 1''', page ) .. .. I see no way of messing with the above statement other that tweak with the 'page' variable but its not clear to me how. You as more experience can you tell how the aboev code of database insertio Here is more insight on how i initiate the 'page' variable: == # define how the .html or .python pages are called path = '/home/nikos/public_html/' cgi_path = '/home/nikos/public_html/cgi-bin/' file = form.getfirst('file', 'forbidden') # this value should come only from .htaccess and not as http://superhost.gr/~nikos/cgi-bin/metrites.py page = form.getvalue('page') # this value comes from 'index.html' or from within 'metrites.py' if os.path.exists( file ) and not page: # it is an html template page = file.replace( path, '' ) == Any ideas please on how the hacker manages to pass arbitrary values into the 'page' var since i explicitly define it and before database insertion i check for: if cookieID != 'nikos' and ( os.path.exists( path + page ) or os.path.exists( cgi_path + page ) ) ?!?! -- https://mail.python.org/mailman/listinfo/python-list
Re: Adding 'download' column to existing 'visitors' table (as requested)
Ah great!!! I just examined my other MySQL database which just stored webpages and their corresponding visits and voila. Someone was able to pass values into my counters table: look: http://superhost.gr/?show=stats thats why it didn't had 1 or 2 or 3 as 'counterID' but more values were present. Someone successfully manipulated this part of my code: if cookieID != 'nikos' and ( os.path.exists( path + page ) or os.path.exists( cgi_path + page ) ) and re.search( r'(amazon|google|proxy|cloud|reverse|fetch|msn|who|spider|crawl|ping)', host ) is None: try: # if first time for webpage; create new record( primary key is automatic, hit is defaulted ), if page exists then update record cur.execute('''INSERT INTO counters (url) VALUES (%s) ON DUPLICATE KEY UPDATE hits = hits + 1''', page ) .. .. I see no way of messing with the above statement other that tweak with the 'page' variable but its not clear to me how. You as more experience can you tell how the aboev code of database insertio -- https://mail.python.org/mailman/listinfo/python-list
Adding 'download' column to existing 'visitors' table (as requested)
I have decided to take your advice. I wasn't able to fit those 'lists' of mine into MySQL's varchar() datatype after converting them to long strings and that sads me. My implementation is like the following. I do not use an extra table of downlaods that i asoociate with table visitors with a foreing key but decided to add an additional 'download' column into the existant visitors table: Here it is: = # ~ DATABASE INSERTS ~ = try: # if first time for webpage; create new record( primary key is automatic, hit is defaulted ), if page exists then update record cur.execute('''INSERT INTO counters (url) VALUES (%s) ON DUPLICATE KEY UPDATE hits = hits + 1''', page ) # get the primary key value of the new added record cID = cur.lastrowid # add this visitor entry into database (hits && downloads are defaulted) cur.execute('''INSERT INTO visitors (counterID, refs, host, city, useros, browser, visits) VALUES (%s, %s, %s, %s, %s, %s, %s)''', (cID, ref, host, city, useros, browser, lastvisit) ) con.commit() except pymysql.ProgrammingError as e: print( repr(e) ) con.rollback() sys.exit(0) = = # ~ Presentation Time = def coalesce( data ): newdata = [] seen = {} for host, refs, city, useros, browser, visits, hits, downloads in data: # Here i have to decide how to group the rows together # I want an html row for every unique combination of (host) and that hits should be summed together key = host if key not in seen: newdata.append( [ host, [refs], city, useros, browser, [visits], hits, [downloads] ] ) seen[key] = len( newdata ) - 1 # Save index (for 'newdata') of this row else: # This row is a duplicate row with a different referrer && visit time && torrent download rowindex = seen[key] newdata[rowindex][1].append( refs ) newdata[rowindex][5].append( visits ) newdata[rowindex][6] += hits newdata[rowindex][7].append( downloads ) return newdata cur.execute( '''SELECT host, refs, city, useros, browser, visits, hits, downloads FROM visitors WHERE counterID = (SELECT ID FROM counters WHERE url = %s) ORDER BY visits DESC''', page ) data = cur.fetchall() newdata = coalesce( data ) for row in newdata: (host, refs, city, useros, browser, visits, hits, downloads) = row # Note that 'refs' && 'visits' && 'downloads' are now lists print( '' ) print( ' %s ' % host ) print( '' ) for ref in refs: print( ' %s ' % ref ) print( '' ) for item in (city, useros, browser): print( ' %s ' % item ) print( '' ) for visit in visits: visittime = visit.strftime('%A %e %b, %H:%M') print( ' %s ' % visittime ) print( '' ) print( ' %s ' % hits ) # populate torrent list torrents = [] for download in downloads: if download: torrents.append( download ) # present visitor's movie picks if any if torrents: print( '' ) for torrent in torrents: print( ' %s ' % torrent ) print( '' ) else: print( ' Δεν πραγματοποίηθηκαν ακόμη! ' ) break print( '' ) sys.exit(0) = At least my webpage http://superhost.gr is working now, but i look into by lookinto into phpmyadmin whats into the database and what is being presented somethign is worng. This is a screenshot of my database visit sicne l