Date: Thu, 17 May 2007 11:13:42 +0930 From: Greg 'groggy' Lehey To: helpdesk@ticket.internode.com.au Subject: IMMEDIATE ACTION REQUESTED (was: [ticket.internode.com.au #782352] AutoReply: Ethereal trace files of SIP routing problems) On Tuesday, 15 May 2007 at 13:00:57 +0930, helpdesk@ticket.internode.com.au wrote: > > Greetings, > > This message has been automatically generated in response to the > creation of a ticket regarding: > "Ethereal trace files of SIP routing problems", > a summary of which appears below. This is, so far, the only written response I have had to this problem. I have the distinct impression that a number of miscommunications have occurred here, so here's a summary: - I'm currently running a class C netblock (192.109.197.0/24) over an IP/IP tunnel. The modem is an AG241 running in bridged mode, and the tunnel is terminated by a FreeBSD box. Outgoing traffic does not go through the tunnel. - I have two Sipura SPA 3000s in this net block. - On 15 May in the morning I returned from a long weekend and discovered that my ADSL line had been upgraded to "unlimited ADSL 1" speed, about 3.5 Mb/s. - I also discovered that my NodePhone line was no longer working. tcpdump showed that registration was working normally, but call setup messages were not being replied to. - I also had problems with the other SPA 3000, which is connected to mynetfone. Communication was usually possible with this device, but not always. - I called tech support and spoke to Dominic. During about an hour of examination, we established: - That this was not a firewall problem (which would have been difficult when some packets get through and others with the same IP addresses and port numbers did not). On the one hand I was looking at the traffic between the firewall and the ADSL line, and to be completely sure I modified the firewall rules to enable all traffic to the SPA 3000 without restriction. - That the traffic seen on my ADSL line did not correspond with the traffic seen at Internode. Both of these could point to some weird routing problem, possibly based on packet size. I've seen related problems in the past (see ticket #704716). - The following morning I had not heard anything. I tried to make a call and was successful, but the connection quality was terrible. This certainly refutes the notion that this is a firewall problem. - After this, nothing happened until I got a call from Jordan on an unrelated issue (transfer of IP/IP tunnel to conventional routing, which you call a "boxed route"). He checked the ticket and found that level 2 was claiming that this was a CPE or firewall problem. Apart from being incorrect, this points at least to miscommunication in two places: - Dominic had already established that this is not the case. - I had not been informed of this conclusion. I reported the marginal attempt to Jordan and asked him to ensure that this problem was fixed. - Yesterday evening at 19:39, long after CoB, I received a phone call from Nick, telling me that the problem was at my end. I tried to explain some of the above, but I got the impression he was either not listening or didn't understand. I don't see how Nick can come to the conclusions he does from the traces I sent. Since those traces, I have installed an additional interface in the FreeBSD gateway, so I can now show you an example (attached, officephone.2) which shows only the traffic across the ADSL line, thus eliminating the duplicates you saw in the previous trace, along with any notion that the firewall could be involved. As a result, this shows the IP/IP traffic, and includes a lot of junk not intended for this trace. Here's a summary: 10:50:05.246579 IP adsl-tunnel-loopback.internode.on.net > ext-gw.lemis.com: IP freefall.freebsd.org > officephone.lemis.com: ICMP echo request, id 55274, seq 166, length 64 (ipip-proto-4) 10:50:05.247731 IP officephone.lemis.com > freefall.freebsd.org: ICMP echo reply, id 55274, seq 166, length 64 10:50:06.250606 IP adsl-tunnel-loopback.internode.on.net > ext-gw.lemis.com: IP freefall.freebsd.org > officephone.lemis.com: ICMP echo request, id 55274, seq 167, length 64 (ipip-proto-4) 10:50:06.251747 IP officephone.lemis.com > freefall.freebsd.org: ICMP echo reply, id 55274, seq 167, length 64 A number of pings from freefall.freebsd.org to ensure that the filter is working correctly. 10:50:19.003623 IP officephone.lemis.com.5060 > sip.internode.on.net.5060: SIP, length: 1016 10:50:19.495058 IP officephone.lemis.com.5060 > sip.internode.on.net.5060: SIP, length: 1016 10:50:20.495617 IP officephone.lemis.com.5060 > sip.internode.on.net.5060: SIP, length: 1016 10:50:22.495168 IP officephone.lemis.com.5060 > sip.internode.on.net.5060: SIP, length: 1016 10:50:26.496619 IP officephone.lemis.com.5060 > sip.internode.on.net.5060: SIP, length: 1016 10:50:34.495575 IP officephone.lemis.com.5060 > sip.internode.on.net.5060: SIP, length: 1016 6 INVITE packets from officephone. 10:50:38.434749 IP officephone.lemis.com.5060 > sip.internode.on.net.5060: SIP, length: 784 10:50:38.934302 IP officephone.lemis.com.5060 > sip.internode.on.net.5060: SIP, length: 784 10:50:39.123976 IP officephone.lemis.com.5060 > sip.internode.on.net.5060: SIP, length: 394 10:50:39.622331 IP officephone.lemis.com.5060 > sip.internode.on.net.5060: SIP, length: 394 10:50:39.934859 IP officephone.lemis.com.5060 > sip.internode.on.net.5060: SIP, length: 784 10:50:40.622364 IP officephone.lemis.com.5060 > sip.internode.on.net.5060: SIP, length: 394 10:50:41.934423 IP officephone.lemis.com.5060 > sip.internode.on.net.5060: SIP, length: 784 10:50:42.622441 IP officephone.lemis.com.5060 > sip.internode.on.net.5060: SIP, length: 394 10:50:45.934542 IP officephone.lemis.com.5060 > sip.internode.on.net.5060: SIP, length: 784 10:50:46.622535 IP officephone.lemis.com.5060 > sip.internode.on.net.5060: SIP, length: 394 10:50:49.934710 IP officephone.lemis.com.5060 > sip.internode.on.net.5060: SIP, length: 784 10:50:50.622729 IP officephone.lemis.com.5060 > sip.internode.on.net.5060: SIP, length: 394 A mixture of REGISTER (784 bytes) and CANCEL (394 bytes) packets. 10:50:53.816633 IP adsl-tunnel-loopback.internode.on.net > ext-gw.lemis.com: IP sip.internode.on.net.5060 > officephone.lemis.com.5060: SIP, length: 427 (ipip-proto-4) 10:50:53.836248 IP adsl-tunnel-loopback.internode.on.net > ext-gw.lemis.com: IP sip.internode.on.net.5060 > officephone.lemis.com.5060: SIP, length: 427 (ipip-proto-4) 10:50:53.839188 IP adsl-tunnel-loopback.internode.on.net > ext-gw.lemis.com: IP sip.internode.on.net.5060 > officephone.lemis.com.5060: SIP, length: 427 (ipip-proto-4) 10:50:53.840927 IP adsl-tunnel-loopback.internode.on.net > ext-gw.lemis.com: IP sip.internode.on.net.5060 > officephone.lemis.com.5060: SIP, length: 427 (ipip-proto-4) 10:50:53.843634 IP adsl-tunnel-loopback.internode.on.net > ext-gw.lemis.com: IP sip.internode.on.net.5060 > officephone.lemis.com.5060: SIP, length: 427 (ipip-proto-4) 10:50:53.846127 IP adsl-tunnel-loopback.internode.on.net > ext-gw.lemis.com: IP sip.internode.on.net.5060 > officephone.lemis.com.5060: SIP, length: 427 (ipip-proto-4) 10:50:53.849349 IP adsl-tunnel-loopback.internode.on.net > ext-gw.lemis.com: IP sip.internode.on.net.5060 > officephone.lemis.com.5060: SIP, length: 427 (ipip-proto-4) The first response packets from sip.internode.on.net, status 401 (unauthorized). The first packet comes 34 seconds after the first INVITE request. 10:50:53.935053 IP officephone.lemis.com.5060 > sip.internode.on.net.5060: SIP, length: 784 REGISTER 10:50:54.011726 IP adsl-tunnel-loopback.internode.on.net > ext-gw.lemis.com: IP sip.internode.on.net.5060 > officephone.lemis.com.5060: SIP, length: 360 (ipip-proto-4) 200 OK 10:50:54.623387 IP officephone.lemis.com.5060 > sip.internode.on.net.5060: SIP, length: 394 CANCEL 10:50:54.660936 IP adsl-tunnel-loopback.internode.on.net > ext-gw.lemis.com: IP sip.internode.on.net.5060 > officephone.lemis.com.5060: SIP, length: 320 (ipip-proto-4) 481 Call/Transaction does not exist. You can, of course, see a lot more with Wireshark. I hope this exhaustive analysis has cleared any confusion about whether the problem is at my end. I'd like to express my disappointment, however, that two days after reporting the problem, I've had to spend over an hour doing the work you should be doing. Please contact me immediately (during normal business hours) and tell me how you intend to proceed. I expect you to fix this problem today. Greg -- Finger grog@lemis.com for PGP public key. See complete headers for address and phone numbers.