Third day in Addis

eng etiopia pdo tips

Believe it or not, a network that fails often is the best thing to have when you are teaching network troubleshooting.

Various tools useful for networking:

ifconfig - configure a network interface
dnsmasq - Simple DNS and DHCP server
host - DNS lookup utility
route - show / manipulate the IP routing table
arping - send ARP REQUEST to a neighbour host
mii-tool - view, manipulate media-independent interface status (IOW, see if the cable works)
nmap - Network exploration tool and security / port scanner

Examples:

    # Look at what machines are active in the local network:
    nmap -sP 10.5.15.0/24

    # Look at what ports are open in a machine:
    nmap 10.5.15.26

tcpdump - dump traffic on a network

It can be used to see if there is traffic, and to detect traffic that shouldn't be there.

Useful tip:

    # Convert a unix timestamp to a readable date
    date -d @1152841341

What happens when you browse a web page:

type the address www.google.com in the browser
the browser needs the IP address of the web server:
look for the DNS address in /etc/resolv.conf (/etc/resolv.conf is created automatically by the DHCP client)
try all the DNS servers in /etc/resolv.conf until one gives you the IP address of www.google.com
take the first address that comes from the DNS (in our case was 64.233.167.104)
figure out how to connect to 64.233.167.104:
consult the routing table to see if it's in the local network:
1. if it's in the local network, then look for the MAC address (using ARP
  - Address Resolution Protocol)
2. if it'd not in the local network, then send through the gateway (again using ARP to find the MAC address of the gateway)
Send out the HTTP request to the local web server or through the gateway, using the Ethernet physical protocol, and the MAC address to refer to the other machine.

Troubleshooting network problems:

See if the network driver works:
With ifconfig, see if you see the HWaddr:. If you do not see it, then the linux driver for the network card is not working. Unfortunately there's no exact way to say that it works perfectly
See if you have an IP address with ifconfig. If you find out that you need to rerun DHCP (for example, if the network cable was disconnected when the system started), then you can do it either by deactivating/reactivating the Ethernet interface using System/Administration/Networking or, on a terminal, running:
```
# ifdown eth0
# ifup eth0
```
If you don't get an IP, try to see if the DHCP server is reachable by running:
```
$ arping -D [address of DHCP server]
```
See if the local physical network works:
With sudo mii-tool, see if the cable link is ok. If it's not, then it's a problem in the cable or the plugs, or simply the device at the other end of the cable is turned off.
Try arping or ping -n on a machine in the local network (like the gateway) to see if the local network works.
See if the DNS works:
Find out the DNS address:
```
cat /etc/resolv.conf
```
If it's local, arping it
If it's not local, ping -n it
Try to resolve a famous name using that DNS:
```
$ host [name] [IP address of the DNS]
```
Try to resolve the name of the machine you're trying to connect. If you can resolve a famous name but not the name you need, then it's likely a problem with their DNS.
If you use a proxy, see if the proxy is reachable: check if the proxy name resolves to an IP, if you can ping it, if you can telnet to the proxy address and port:
```
$ telnet [proxy address] [proxy port]
```
you quit telnet with ^]quit.
If you can connect directly to the web server, try to see if it answers:
```
$ telnet [address] 80
```
If you are connected, you can confirm that it's a web server:
```
GET / HTTP/1.0 (then Enter twice)
```
If it's a web server, it should give you something like a webpage or an HTTP redirect.

When you try to setup a service and it doesn't work:

check that it's running:
```
$ ps aux | grep dnsmasq
```
check that it's listening on the right port:
```
$ sudo netstat -lp
```
check that it's listening from the outside:
```
$ nmap [hostname]
```
check for messages in /var/log/daemon.log or /var/log/syslog
check that the configuration is correct and reload or restart the server to make sure it's running with the right configuration:
```
# /etc/init.d/dnsmasq restart
```

dnsmasq:

By default: works as a DNS server that serves the data in /etc/hosts.

By default: uses /etc/resolv.conf to find addresses of other DNS to use when a name is not found in /etc/hosts.

To enable the DHCP server, uncomment:

    dhcp-range=192.168.0.50,192.168.0.150,12h

in /etc/dnsmasq.conf and set it to the range of addresses you want to serve. Pay attention to never put two DHCP servers on the same local network, or they will interfere with each others.

To test if the DHCP server is working, use dhcping (not installed by default on Ubuntu).

To communicate other information like DNS, gateway and netmask to the clients, use this piece of dnsmasq.conf:

    # For reference, the common options are:
    # subnet mask - 1
    # default router - 3
    # DNS server - 6
    # broadcast address - 28
    dhcp-option=1,255.255.255.0
    dhcp-option=3,192.168.0.1
    dhcp-option=6,192.168.0.1
    dhcp-option=28,192.168.0.255

Problems found today:

changing the name of the local machine in /etc/hosts breaks sudo, and without sudo it's impossible to edit the file. The only way to fix this is a reboot in recovery mode.
dhclient -n -w is different than dhclient -nw

Quick start examples with tar:

    # Create an archive
    tar zcvf nmap.tar.gz *.deb

    # Extract an archive
    tar zxvf nmap.tar.gz

    # Look at the contents of an archive
    tar ztvf nmap.tar.gz

Quick & dirty way to send a file between two computers without web server, e-mail, shared disk space or any other infrastructure:

    # To send
    nc -l -p 12345 -q 1 < nmap.tar.gz

    # To receive
    nc 10.5.15.123 12345 > nmap.tar.gz

    # To repeat the send command 20 times
    for i in `seq 1 20`; do nc -l -p 12345 -q 1 < nmap.tar.gz ; done

Update: Javier Fernandez-Sanguino writes:

Your "XXX day in Addis" is certainly good reading, nice to see somebody reviewing common tools from a novice point of view. Some comments:

Regarding your comments on how to troubleshoot network connectivity problems I just wanted to point you to the network test script I wrote and submited to the debian-goodies package ages ago. It's available at http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=307694 and should do automatically most of the stuff you commented on your blog.

Your example to test hosts alive in the network using nmap -sP 10.5.15.0/24 is good. However, newer (v4) versions can do ARP ping in the local network which is much more efficient (some systems might block ICMP outbount), that's the -PR option and should be enabled (by default). See http://www.insecure.org/nmap/man/man-host-discovery.html Also, you might want to add a '-n' there so that nmap does not try to do DNS resolution of the hosts (which might take up some time if your DNS does not include local IPs)

tcpdump, it would be wiser to turn novice users to ethereal since it has a much better UI than tcpdump and it is able to dissect (interpret) protocols that tcpdump can't analyse.

you are missing arp as a tool in itself, it is useful to debug network issues since if the host is local and does not show up in arp output either a) it's down or b) you don't have proper network connectivity. (If you are missing an ARP entry for your default gateway your setup is broken)

Update: Marius Gedminas writes:

Re: http://www.enricozini.org/blog/eng/third-day-in-addis

In my experience if sudo cannot resolve the hostname (e.g. if you break /etc/hosts), you can still use sudo, but you have to wait something like 30 seconds until the DNS request times out.

I tried to break my /etc/hosts (while keeping a root shell so I can fix it if something goes wrong), but couldn't even get the timeout now. Sudo just said unable to lookup $hostname via gethostbyname() and gave me a root shell.