Network troubleshooting, any experts?

Wednesday, 28 April 2004

Hi list,

First let me say that the particular server that I am trying to 
troubleshoot is not Fedora, but
RedHat 9.  As I am subscribed to this list, I thought it would be a good 
place to ask.

At this particular company we have a webserver, that sits behind a 
firewall/router.  All incoming port 80
traffic is directed to this server.  All computers in the company reside 
internally on 123.123.123.* ip addresses.
All DNS resolution is done externally.

Now the problem is that all computers on the network can browse the 
internet and do various chores like
telnet and ssh with no problem, except for the web server.  I can ssh, 
telnet etc. to other computers on the internal network
from the web server but not to the outside world.

Some oddities:

My resolv.conf file has the ip addresses of my DNS servers.  If I ping 
an internet address I get back the ip resolution ok, yet I cannot
telnet to either of my DNS servers on port 53 from the web server.  So 
how am I getting back ip addresses when I ping?

Traceroute and ping respond ok, but no other utilities respond.  They 
all time out.  I ran some tcpdump's telneting to yahoo.com and the DNS 
server and I've
included those below if it's helpful to anyone.

I have no firewall running, and just to be sure I've flushed the 
iptables and ran the /etc/rc3.d/iptables script with the -stop flag.
I've also talked to the isp( it's their router ) and they claim that if 
all the other computers can get web access then so should
the webserver.

If anyone has ANY suggestions it would be most helpful.

Cheers,
Elam Daly
Whiteware Inc.

TCPDUMP to YAHOO.COM telnet port 80:

15:20:05.621044 123.123.123.240.1065 > sprite.wwnet.net.domain:  29834+ 
AAAA? www.yahoo.com. (31) (DF)
15:20:05.700534 sprite.wwnet.net.domain > 123.123.123.240.1065:  29834 
1/1/0 (137)
15:20:05.700874 123.123.123.240.1065 > sprite.wwnet.net.domain:  29835+ 
A? www.yahoo.com. (31) (DF)
15:20:05.723337 sprite.wwnet.net.domain > 123.123.123.240.1065:  29835 
9/9/9 CNAME[|domain]
15:20:05.724132 123.123.123.240.1065 > sprite.wwnet.net.domain:  1558+ 
PTR? 68.118.109.216.in-addr.arpa. (45) (DF)
15:20:05.830093 sprite.wwnet.net.domain > 123.123.123.240.1065:  1558* 
1/5/5 (276)
15:20:05.830519 123.123.123.240.1065 > sprite.wwnet.net.domain:  1559+ 
PTR? 65.118.109.216.in-addr.arpa. (45) (DF)
15:20:05.893671 sprite.wwnet.net.domain > 123.123.123.240.1065:  1559* 
1/5/5 (276)
15:20:05.894048 123.123.123.240.1065 > sprite.wwnet.net.domain:  1560+ 
PTR? 108.117.109.216.in-addr.arpa. (46) (DF)
15:20:06.000311 sprite.wwnet.net.domain > 123.123.123.240.1065:  1560* 
1/5/5 (279)
15:20:06.000687 123.123.123.240.1065 > sprite.wwnet.net.domain:  1561+ 
PTR? 70.118.109.216.in-addr.arpa. (45) (DF)
15:20:06.060732 sprite.wwnet.net.domain > 123.123.123.240.1065:  1561* 
1/5/5 (276)
15:20:06.061147 123.123.123.240.1065 > sprite.wwnet.net.domain:  1562+ 
PTR? 73.118.109.216.in-addr.arpa. (45) (DF)
15:20:06.199215 sprite.wwnet.net.domain > 123.123.123.240.1065:  1562* 
1/5/5 (277)
15:20:06.199595 123.123.123.240.1065 > sprite.wwnet.net.domain:  1563+ 
PTR? 66.118.109.216.in-addr.arpa. (45) (DF)
15:20:06.256277 sprite.wwnet.net.domain > 123.123.123.240.1065:  1563* 
1/5/5 (276)
15:20:06.256652 123.123.123.240.1065 > sprite.wwnet.net.domain:  1564+ 
PTR? 74.118.109.216.in-addr.arpa. (45) (DF)
15:20:06.320372 sprite.wwnet.net.domain > 123.123.123.240.1065:  1564* 
1/5/5 (277)
15:20:06.320748 123.123.123.240.1065 > sprite.wwnet.net.domain:  1565+ 
PTR? 205.117.109.216.in-addr.arpa. (46) (DF)
15:20:06.383390 sprite.wwnet.net.domain > 123.123.123.240.1065:  1565* 
1/5/5 (279)
15:20:06.384242 123.123.123.240.3796 > p5.www.dcn.yahoo.com.http: S 
1740721116:1740721116(0) win 5840 <mss 1460,sackOK,timestamp 8969937 
0,nop,wscale 0> (DF) [tos 0x10]
15:20:09.375214 123.123.123.240.3796 > p5.www.dcn.yahoo.com.http: S 
1740721116:1740721116(0) win 5840 <mss 1460,sackOK,timestamp 8970237 
0,nop,wscale 0> (DF) [tos 0x10]
15:20:15.375192 123.123.123.240.3796 > p5.www.dcn.yahoo.com.http: S 
1740721116:1740721116(0) win 5840 <mss 1460,sackOK,timestamp 8970837 
0,nop,wscale 0> (DF) [tos 0x10]

TCPDUMP to DNS SERVER, telnet port 53:

15:28:23.096096 123.123.123.240.1066 > sprite.wwnet.net.domain:  32519+ 
AAAA? sprite.wwnet.net. (34) (DF)
15:28:23.115363 sprite.wwnet.net.domain > 123.123.123.240.1066:  32519* 
0/1/0 (85)
15:28:23.115706 123.123.123.240.1066 > sprite.wwnet.net.domain:  32520+ 
AAAA? sprite.wwnet.net.localdomain. (46) (DF)
15:28:23.134217 sprite.wwnet.net.domain > 123.123.123.240.1066:  32520 
NXDomain 0/1/0 (121)
15:28:23.134782 123.123.123.240.1066 > sprite.wwnet.net.domain:  32521+ 
A? sprite.wwnet.net. (34) (DF)
15:28:23.154865 sprite.wwnet.net.domain > 123.123.123.240.1066:  32521* 
1/2/2 A sprite.wwnet.net (119)
15:28:23.155665 123.123.123.240.1066 > sprite.wwnet.net.domain:  21669+ 
PTR? 2.211.142.209.in-addr.arpa. (44) (DF)
15:28:23.176607 sprite.wwnet.net.domain > 123.123.123.240.1066:  21669* 
1/2/2 (143)
15:28:23.177382 123.123.123.240.3799 > sprite.wwnet.net.domain: S 
2259943146:2259943146(0) win 5840 <mss 1460,sackOK,timestamp 9019617 
0,nop,wscale 0> (DF) [tos 0x10]
15:28:26.175190 123.123.123.240.3799 > sprite.wwnet.net.domain: S 
2259943146:2259943146(0) win 5840 <mss 1460,sackOK,timestamp 9019917