Re: [Tutor] Socket Module
Writing to share an update on my previous request.
So, after reviewing my code over, it seems like my last print statement
"print(ResolvedAddresses)" was not properly indented inside the for loop - and
for that reason the output was coming our as empty.
After I aligned that with the rest of the loop, it worked fine.
I still have couple parts to add to this project but at least this hurdle is
behind.
Thank you.
On Tuesday, July 28, 2015 4:49 AM, Nym City via Tutor
wrote:
Hi Martin,
Thank you for taking the time to write such a detailed response. Very much
appreciate it.
I tried the code again with modifications that you suggested and even though
none of the public addresses resolved; I did get little more details.
I am still working on finding a solution for this and I will share it here when
I have it.
Thank you.
On Sunday, July 26, 2015 6:59 PM, Martin A. Brown
wrote:
Hello Nym,
> Here is the updated code: https://bpaste.net/show/358583e1a0bd
It's short. I have included inline here:
import socket
ListOfIPAddresses = []
with open('top500ips.csv', 'r') as f:
for line in f:
line = line.strip()
ListOfIPAddresses.append(line)
for address in ListOfIPAddresses:
try:
ResolvedAddresses = socket.gethostbyaddr(address)
except:
print('No Resolution Available')
print(ResolvedAddresses)
> The issue that I am running into now is that for some reason, the
> script is not resolving known-public IP addresses that I am
> passing through. For example, some of the IPs, that I have used
> are for sites like facebook (173.252.120.6) github
> (207.97.227.239), however the script is not able to resolve them.
>
> But its interesting that I am able to resolve them using nslookup
> on windows command prompt. Which means my laptop's DNS setting is
> fine.
The apparent (?) DNS lookup failure
---
At time X, you run your Python program and something (perhaps in the
DNS resolution process) fails and you see "No Resolution Available",
but you do not know what has failed, nor for which address lookup.
At time Y, you run 'nslookup' at the shell prompt, receive an answer
and conclude that your script is operating properly. While this is
may appear logical, it is an incorrect conclusion.
One coding error (a dangerous habit to perpetuate)
--
When performing the DNS lookup, you are using something called
a 'bare except'. This will catch any and all errors, even if it's
something unrelated like, for example, a signal. This is a bad and
dangerous habit. In general, you should catch only the exceptions
that you can do something about.
In addition, this will offer you more information about the problem.
Here's a simple example, where I'm only changing two lines:
for address in ListOfIPAddresses:
try:
ResolvedAddresses = socket.gethostbyaddr(address)
except socket.herror as e:
print("No resolution available for %s: %s" % (address, e))
This will give you a little more information about what,
specifically, the failure is in your call to socket.gethostbyaddr()
Comment on NXdomain responses
-
I picked at random an address that had no PTR record and tried to
call socket.gethostbyaddr('8.97.227.2'). What do you think I got
for my trouble? When running through the code block above, I saw
the following output to my terminal:
No resolution available for 8.97.227.2: [Errno 0] Resolver Error 0 (no error)
In short, there is no guarantee that anybody has properly set up
reverse DNS entries (DNS PTR records) for the addresses you are
looking up. Although the vast majority of lookups occur
successfully and smoothly, there are many things that can go wrong
in the network and on an end host which can cause transient errors
during DNS lookups, and it is possible that you have already
encountered some of these problems (even though I would not expect
to hit very many errors looking up PTR records for only 500 IPs).
May I wish you good luck resolving not just your addresses, but also
your problem!
-Martin
--
Martin A. Brown
http://linux-ip.net/
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Socket Module
Hi Martin,
Thank you for taking the time to write such a detailed response. Very much
appreciate it.
I tried the code again with modifications that you suggested and even though
none of the public addresses resolved; I did get little more details.
I am still working on finding a solution for this and I will share it here when
I have it.
Thank you.
On Sunday, July 26, 2015 6:59 PM, Martin A. Brown
wrote:
Hello Nym,
> Here is the updated code: https://bpaste.net/show/358583e1a0bd
It's short. I have included inline here:
import socket
ListOfIPAddresses = []
with open('top500ips.csv', 'r') as f:
for line in f:
line = line.strip()
ListOfIPAddresses.append(line)
for address in ListOfIPAddresses:
try:
ResolvedAddresses = socket.gethostbyaddr(address)
except:
print('No Resolution Available')
print(ResolvedAddresses)
> The issue that I am running into now is that for some reason, the
> script is not resolving known-public IP addresses that I am
> passing through. For example, some of the IPs, that I have used
> are for sites like facebook (173.252.120.6) github
> (207.97.227.239), however the script is not able to resolve them.
>
> But its interesting that I am able to resolve them using nslookup
> on windows command prompt. Which means my laptop's DNS setting is
> fine.
The apparent (?) DNS lookup failure
---
At time X, you run your Python program and something (perhaps in the
DNS resolution process) fails and you see "No Resolution Available",
but you do not know what has failed, nor for which address lookup.
At time Y, you run 'nslookup' at the shell prompt, receive an answer
and conclude that your script is operating properly. While this is
may appear logical, it is an incorrect conclusion.
One coding error (a dangerous habit to perpetuate)
--
When performing the DNS lookup, you are using something called
a 'bare except'. This will catch any and all errors, even if it's
something unrelated like, for example, a signal. This is a bad and
dangerous habit. In general, you should catch only the exceptions
that you can do something about.
In addition, this will offer you more information about the problem.
Here's a simple example, where I'm only changing two lines:
for address in ListOfIPAddresses:
try:
ResolvedAddresses = socket.gethostbyaddr(address)
except socket.herror as e:
print("No resolution available for %s: %s" % (address, e))
This will give you a little more information about what,
specifically, the failure is in your call to socket.gethostbyaddr()
Comment on NXdomain responses
-
I picked at random an address that had no PTR record and tried to
call socket.gethostbyaddr('8.97.227.2'). What do you think I got
for my trouble? When running through the code block above, I saw
the following output to my terminal:
No resolution available for 8.97.227.2: [Errno 0] Resolver Error 0 (no error)
In short, there is no guarantee that anybody has properly set up
reverse DNS entries (DNS PTR records) for the addresses you are
looking up. Although the vast majority of lookups occur
successfully and smoothly, there are many things that can go wrong
in the network and on an end host which can cause transient errors
during DNS lookups, and it is possible that you have already
encountered some of these problems (even though I would not expect
to hit very many errors looking up PTR records for only 500 IPs).
May I wish you good luck resolving not just your addresses, but also
your problem!
-Martin
--
Martin A. Brown
http://linux-ip.net/
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Socket Module
Hello Nym,
Here is the updated code: https://bpaste.net/show/358583e1a0bd
It's short. I have included inline here:
import socket
ListOfIPAddresses = []
with open('top500ips.csv', 'r') as f:
for line in f:
line = line.strip()
ListOfIPAddresses.append(line)
for address in ListOfIPAddresses:
try:
ResolvedAddresses = socket.gethostbyaddr(address)
except:
print('No Resolution Available')
print(ResolvedAddresses)
The issue that I am running into now is that for some reason, the
script is not resolving known-public IP addresses that I am
passing through. For example, some of the IPs, that I have used
are for sites like facebook (173.252.120.6) github
(207.97.227.239), however the script is not able to resolve them.
But its interesting that I am able to resolve them using nslookup
on windows command prompt. Which means my laptop's DNS setting is
fine.
The apparent (?) DNS lookup failure
---
At time X, you run your Python program and something (perhaps in the
DNS resolution process) fails and you see "No Resolution Available",
but you do not know what has failed, nor for which address lookup.
At time Y, you run 'nslookup' at the shell prompt, receive an answer
and conclude that your script is operating properly. While this is
may appear logical, it is an incorrect conclusion.
One coding error (a dangerous habit to perpetuate)
--
When performing the DNS lookup, you are using something called
a 'bare except'. This will catch any and all errors, even if it's
something unrelated like, for example, a signal. This is a bad and
dangerous habit. In general, you should catch only the exceptions
that you can do something about.
In addition, this will offer you more information about the problem.
Here's a simple example, where I'm only changing two lines:
for address in ListOfIPAddresses:
try:
ResolvedAddresses = socket.gethostbyaddr(address)
except socket.herror as e:
print("No resolution available for %s: %s" % (address, e))
This will give you a little more information about what,
specifically, the failure is in your call to socket.gethostbyaddr()
Comment on NXdomain responses
-
I picked at random an address that had no PTR record and tried to
call socket.gethostbyaddr('8.97.227.2'). What do you think I got
for my trouble? When running through the code block above, I saw
the following output to my terminal:
No resolution available for 8.97.227.2: [Errno 0] Resolver Error 0 (no error)
In short, there is no guarantee that anybody has properly set up
reverse DNS entries (DNS PTR records) for the addresses you are
looking up. Although the vast majority of lookups occur
successfully and smoothly, there are many things that can go wrong
in the network and on an end host which can cause transient errors
during DNS lookups, and it is possible that you have already
encountered some of these problems (even though I would not expect
to hit very many errors looking up PTR records for only 500 IPs).
May I wish you good luck resolving not just your addresses, but also
your problem!
-Martin
--
Martin A. Brown
http://linux-ip.net/
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Socket Module
Thank you all for your responses. I have taken your feedback and made changes
to my code.
-Danny, per your suggestion, I have renamed some of my variables to make their
purpose little more clearer.
- Alan, I have created a new host list (ResolvedAddresses) which is storing
the output from socket.gethostbyaddr(address).
Here is the updated code: https://bpaste.net/show/358583e1a0bd
The issue that I am running into now is that for some reason, the script is not
resolving known-public IP addresses that I am passing through. For example,
some of the IPs, that I have used are for sites like facebook (173.252.120.6)
github (207.97.227.239), however the script is not able to resolve them.
But its interesting that I am able to resolve them using nslookup on windows
command prompt.
Which means my laptop's DNS setting is fine.
Looking forward to your feedback. Thanks!
Thank you.
On Monday, July 20, 2015 5:00 AM, Alan Gauld
wrote:
On 20/07/15 00:55, Nym City via Tutor wrote:
> Thank you for your response. I gave it another try:
> As suggested, first I ran the concept just in the terminal, and it worked
> fine:
names =['173.252.120.6', '98.139.183.24']
import socket
for name in names:
> socket.gethostbyaddr(name)
> print(name)
>
> output:
> ('edge-star-shv-12-frc3.facebook.com', [], ['173.252.120.6'])
> ('ir2.fp.vip.bf1.yahoo.com', [], ['98.139.183.24'])
Remember that the >>> prompt evaluates your results and
automatically prints them for you.
Thus
>>> 5 + 3
8
But if you put code in a script and execute it the interpreter
does NOT print out arbitrary expressions so a file add.py
containing just
5 + 3
will not output anything, you need the print function:
print(5+3)
to see the result.
So in the interpreter your gethostbyaddr() function is
evaluated AND printed. But in a script it will only be
evaluated...
> import csv
> import socket
>
> domains = []
>
> with open('top500ips.csv', 'r') as f:
> for line in f:
> line = line.strip()
> domains.append(line)
You are importing csv but not using it to read your file.
But I'll ignore that for now! (in fact it looks like the
csv file really only contains the IP addresses, one per
line so csv is probably redundant here.)
Also you could do all of the above in a single line with:
domains = [line.strip() for line in open('top500ips.csv')]
But none of that matters for your specific issue...
> for name in domains:
> socket.gethostbyaddr(name)
> print(name)
Notice here that you run the gethostbyaddr() function but
do nothing with the result. You don't store it and you
don't print it. Instead you print the initial name that
you passed in, which does not change.
You need to create a variable to store the host result
and then print that host. And since you want to store a
set of hosts you probably want to append the results to
a list of some kind (or a dictionary based on your names?)
--
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Socket Module
On 20/07/15 00:55, Nym City via Tutor wrote:
Thank you for your response. I gave it another try:
As suggested, first I ran the concept just in the terminal, and it worked fine:
names =['173.252.120.6', '98.139.183.24']
import socket
for name in names:
socket.gethostbyaddr(name)
print(name)
output:
('edge-star-shv-12-frc3.facebook.com', [], ['173.252.120.6'])
('ir2.fp.vip.bf1.yahoo.com', [], ['98.139.183.24'])
Remember that the >>> prompt evaluates your results and
automatically prints them for you.
Thus
>>> 5 + 3
8
But if you put code in a script and execute it the interpreter
does NOT print out arbitrary expressions so a file add.py
containing just
5 + 3
will not output anything, you need the print function:
print(5+3)
to see the result.
So in the interpreter your gethostbyaddr() function is
evaluated AND printed. But in a script it will only be
evaluated...
import csv
import socket
domains = []
with open('top500ips.csv', 'r') as f:
for line in f:
line = line.strip()
domains.append(line)
You are importing csv but not using it to read your file.
But I'll ignore that for now! (in fact it looks like the
csv file really only contains the IP addresses, one per
line so csv is probably redundant here.)
Also you could do all of the above in a single line with:
domains = [line.strip() for line in open('top500ips.csv')]
But none of that matters for your specific issue...
for name in domains:
socket.gethostbyaddr(name)
print(name)
Notice here that you run the gethostbyaddr() function but
do nothing with the result. You don't store it and you
don't print it. Instead you print the initial name that
you passed in, which does not change.
You need to create a variable to store the host result
and then print that host. And since you want to store a
set of hosts you probably want to append the results to
a list of some kind (or a dictionary based on your names?)
--
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Socket Module
> for name in domains: > socket.gethostbyaddr(name) > print(name) > > output: > 173.252.120.6 > 98.139.183.24 > > What am I missing? Thank in advance. > You have confused yourself a little because the variable names you've chosen are slightly misleading. Specifically, "name" is really an IP address. Hence, your loop here really should be: for addr in domains: ... I would also strongly suggest renaming "domains" to something like "domainAddresses", for similar reasons. Still, you do want a "name" at some point, since that's what you care about. Can you say in human language what the following will return? socket.gethostbyaddr(addr) Can you give it a good name? ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Socket Module
Thank you for your response. I gave it another try:
As suggested, first I ran the concept just in the terminal, and it worked fine:
>>> names =['173.252.120.6', '98.139.183.24']
>>> import socket
>>> for name in names:
socket.gethostbyaddr(name)
print(name)
output:
('edge-star-shv-12-frc3.facebook.com', [], ['173.252.120.6'])
('ir2.fp.vip.bf1.yahoo.com', [], ['98.139.183.24'])
However, when I run it in a program, from a CSV file, it just outputs the
content of the CSV file without running it thought socket.gethostbyaddr():
import csv
import socket
domains = []
with open('top500ips.csv', 'r') as f:
for line in f:
line = line.strip()
domains.append(line)
for name in domains:
socket.gethostbyaddr(name)
print(name)
output:
173.252.120.6
98.139.183.24
What am I missing? Thank in advance.
Thank you.
On Saturday, July 18, 2015 7:09 PM, Danny Yoo wrote:
On Jul 18, 2015 3:50 PM, "Nym City via Tutor" wrote:
>
> Thank you all for your responses. I have a follow up question:
>
> So if gethostbyname_ex() takes only a single hostname string, how can I use
> it to go through a list of hostnames and get their IP resolution as an output?
>
Look into loops. If you have a function that works on a single thing, you can
use a loop to apply that function for each element in a list. Any good
tutorial should show how to do this.Let us know if you run into difficulties.
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Socket Module
On Jul 18, 2015 3:50 PM, "Nym City via Tutor" wrote: > > Thank you all for your responses. I have a follow up question: > > So if gethostbyname_ex() takes only a single hostname string, how can I use it to go through a list of hostnames and get their IP resolution as an output? > Look into loops. If you have a function that works on a single thing, you can use a loop to apply that function for each element in a list. Any good tutorial should show how to do this. Let us know if you run into difficulties. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Socket Module
Thank you all for your responses. I have a follow up question:
So if gethostbyname_ex() takes only a single hostname string, how can I use it
to go through a list of hostnames and get their IP resolution as an output?
This would mean,socket.gethostbyaddr() would also not work for my second
project.
Thank you.
On Monday, July 13, 2015 2:59 AM, Alan Gauld
wrote:
On 12/07/15 23:36, Nym City via Tutor wrote:
> import csv
> import socket
>
> domains = open('top500domains.csv', 'r')
> for domain in domains:
> domain = socket.gethostbyname(str(domains))
You are passing your file object to gethostbyname()
> print(domains + "\n")
> For the code above, I receive the following error on run: socket.gaierror:
> [Errno -2] Name or service not known.
> Variation 2:
> import csv
> import socketdomains = []
I assume the domains bit is on a separate line?
>
> with open('top500domains.csv', 'r') as f:
> for line in f:
> line = line.strip()
> domains.append(line)
>
> hostbyname_ex = socket.gethostbyname_ex(str(domains))
Again you are passing the entire list to gethostbyname_ex()
--
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Socket Module
On 12/07/15 23:36, Nym City via Tutor wrote:
import csv
import socket
domains = open('top500domains.csv', 'r')
for domain in domains:
domain = socket.gethostbyname(str(domains))
You are passing your file object to gethostbyname()
print(domains + "\n")
For the code above, I receive the following error on run: socket.gaierror:
[Errno -2] Name or service not known.
Variation 2:
import csv
import socketdomains = []
I assume the domains bit is on a separate line?
with open('top500domains.csv', 'r') as f:
for line in f:
line = line.strip()
domains.append(line)
hostbyname_ex = socket.gethostbyname_ex(str(domains))
Again you are passing the entire list to gethostbyname_ex()
--
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Socket Module
One other thing to note: if you're working with a comma-separated value file (CSV), then you may want to use the 'csv' module to parse it. https://docs.python.org/3.5/library/csv.html This should allow you to walk through the file as if it were a sequence of records. In contrast, if you're dealing with your input as a sequence of lines, then you have more to deal with: you need to think about issues like line endings, which Cameron has pointed out. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Socket Module
On 12Jul2015 22:36, Nym City wrote:
Hello,
I am working on a 2 individual programs. In the first program, I am taking in a
list of domains in .csv format and want to get the associated IPs.
In the second program, I want to import in a list of IP .csv and than get the
associated domains out.
Eventually I want to get both of the above programs to write out to the same
.csv file that they are reading from- in the next column. But I am not there
yet.
Doing research I came across the following
example:http://python.about.com/od/pythonstandardlibrary/qt/dnscheck.htm
From the link above, doing domain -> IP would be easier to begin with. Here are
the two variations of my code:
Variation 1:
import csv
import socket
domains = open('top500domains.csv', 'r')
for domain in domains:
You're getting a whole line here, including the traling newline. Add this line:
print("domain=%r" % (domain,))
at the start of the loop to see this.
domain = socket.gethostbyname(str(domains))
There's no point converting a line to str; a line is a str. If you're wokring
from example code I suppose you might have mistyped "strip".
Anyway, you're passing a string ending in a newline to gethostbyname. It will
not resolve.
print(domains + "\n")
For the code above, I receive the following error on run: socket.gaierror:
[Errno -2] Name or service not known.
As the exception suggests. When debugging issues like these, _always_ put in
print() calls ahead of the failing function to ensure that you are passing the
values you think you're passing.
Variation 2:
import csv
import socketdomains = []
with open('top500domains.csv', 'r') as f:
for line in f:
line = line.strip()
domains.append(line)
hostbyname_ex = socket.gethostbyname_ex(str(domains))
gethostbyname_ex takes a single hostname string. You are passing a list of
strings.
print(hostbyname_ex)
I receive the same error as the first variation.
Please share your advice and let me know what I am doing wrong. The
top500domains.csv list is attached.
Plenty of mailing list software strips attachments. In future, unless the file
is large, just append the contents below your message (with some explaination).
Cheers,
Cameron Simpson
Life is uncertain. Eat dessert first. - Jim Blandy
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
