Dreamhost bouncing LL mail?
|
|
Sapphire Bombay
Avatar
Join date: 8 Oct 2003
Posts: 341
|
11-09-2005 11:38
From: Mark Linden
Internap isn't blocking Dreamhost. I am not blocking Dreamhost. Linden Lab doesn't IP blacklist anybody as policy unless we're under attack.
So if Internap is blocking Dreamhost, how do we get them to stop doing that? Not by calling Dreamhost but by asking Internap to remove the block. Will they do that? Considering it is LL's ISP and it is affecting about 40 of LL's customers, I vote LL call them and ask them to remove the block. Internap San Francisco office # 415-296-8522. I have a choice to continue my Dreamhost annual billing in a few days. I need to know if Dreamhost is going to be a viable web host to use with SL or not.
_____________________
Avatar: A temporary manifestation or aspect of a continuing entity.
|
|
Krazzora Zaftig
Do you have my marbles?
Join date: 20 Aug 2005
Posts: 649
|
11-09-2005 12:35
sapphire Internap is NOT blocking anyone. But for the sake of arguement here is the flowchart of who goes to who.
Players go to LL or Dreamhost
LL goes to Players or Internap
Dream Host goes to Players or Level 3
In Abuse situations once we find out who is blocking who then those two talk as well.
EDIT: To answer your question about Dream Host being viable or not...you'll need to talk to Dream Host and ask them. Most likly...I'd leave and tell them why.
|
|
Sapphire Bombay
Avatar
Join date: 8 Oct 2003
Posts: 341
|
11-09-2005 12:52
Basically it comes down to finger pointing and nobody from either company caring about it. Dreamhost says that LL's ISP is blocking. LL says that is not true. From the trace routes I see I tend to believe Dreamhost on that one (in fact it is this device: border5.fe5-23.linden-1.sfo.pnap.net (66.150.245.252)). Either way wouldn't you think LL would take a vested interest in helping us resolve this?
_____________________
Avatar: A temporary manifestation or aspect of a continuing entity.
|
|
Krazzora Zaftig
Do you have my marbles?
Join date: 20 Aug 2005
Posts: 649
|
11-09-2005 13:18
Sapphire
In my experience LL is doing all it can to help. They have internal documentation and know THEY are not blocking DreamHost. LL has spoken with thier ISP Internap and THEIR internal documentation shows they are not blocking DreamHost. DreamHost and Level 3 need to be checked. LL has contacted DreamHost's abuse department to see if they are blocking (often abuse related groups will speak only to the one blocked) and are waiting on a reply. From there Level 3 might need to be contacted and either LL or DreamHost will do it. It is a slow, LEGAL problem and these departments move carefully. An example is that abuse departments have to deal with companies like Level 3, AOL, Yahoo, Google, etc. Obviously you say the wrong thing or anyone in the company does! and there is going to be issues.
EDIT: Abuse departments also usually work with local authorities, FBI, and CIA against all sorts of illegal acts like MP3/Movie downloads, illegal erotic photography, virii/spam emails, and hacker attacks.
|
|
Sapphire Bombay
Avatar
Join date: 8 Oct 2003
Posts: 341
|
11-09-2005 13:28
Here it is as simply as I can put it: The trace route from DH to LL fails one step past here: border1.ge1-1-bbnet1.sfo002.pnap.net (63.251.63.1) (see step 7 of first trace below) The trace route from LL to DH does not fail and shows that the device past the failure is: border5.fe5-23.linden-1.sfo.pnap.net (66.150.245.252) (see step 2 of second trace below) pnap.net is Internap and is LL's ISP The packets are getting filtered one hop past 63.251.63 subnet. That device is sitting at Linden's data center based on the naming. Which means nobody but LL is affected by this block. LL should simply be able to call Internap and ask them to remove the filter on border5.fe5-23.linden-1.sfo.pnap.net and any alternate path routers. Dreamhost to Linden LabsFrom: someone [slimy]$ traceroute lindenlab.com traceroute: Warning: lindenlab.com has multiple addresses; using 66.150.244.150 traceroute to lindenlab.com (66.150.244.150), 30 hops max, 38 byte packets 1 gw-66-33-192-1 (66.33.192.1) 0.390 ms 0.814 ms 1.959 ms 2 gw-L3 (4.78.192.65) 1.604 ms 0.558 ms 4.365 ms 3 ae-1-56.bbr2.LosAngeles1.Level3.net (4.68.102.161) 4.475 ms ae-1-54.bbr2.LosAngeles1.Level3.net (4.68.102.97) 0.843 ms ae-1-52.bbr2.LosAng eles1.Level3.net (4.68.102.33) 1.191 ms 4 so-3-0-0.mp1.SanFrancisco1.Level3.net (209.247.8.89) 13.459 ms 13.857 ms as-0-0.mp2.SanFrancisco1.Level3.net (64.159.0.217) 13.151 ms 5 ge-7-0-0.gar1.SanFrancisco1.Level3.net (4.68.124.210) 17.424 ms 14.708 ms 13.216 ms 6 4.78.242.18 (4.78.242.1  14.460 ms 14.194 ms 14.773 ms 7 border1.ge1-1-bbnet1.sfo002.pnap.net (63.251.63.1) 12.199 ms border1.ge2-1-bbnet2.sfo002.pnap.net (63.251.63.65) 14.157 ms border1.ge1-1-b bnet1.sfo002.pnap.net (63.251.63.1) 12.220 ms 8 * * * From: someone [mark@ultra ~]$ traceroute slimy.dreamhost.com 1 border5.fe5-23.linden-1.sfo.pnap.net (66.150.245.252) 0.369 ms 0.282 ms 0.226 ms 2 core1.ae1-bbnet2.sfo002.pnap.net (63.251.63.84) 0.323 ms 0.366 ms 0.290 ms3 sl-gw19-sj-10-0-1.sprintlink.net (144.228.111.93) 71.748 ms 2.753 ms 2.654 ms 4 sl-bb20-sj-4-0.sprintlink.net (144.232.0.225) 2.343 ms 2.268 ms 2.200 ms 5 sl-bb21-sj-15-0.sprintlink.net (144.232.3.15 2.185 ms 2.152 ms 2.223 ms 6 sl-st20-sj-13-0.sprintlink.net (144.232.9.5 2.637 ms 2.654 ms 2.593 ms 7 so-7-1.car4.SanJose1.Level3.net (209.245.146.245) 2.294 ms 2.321 ms 2.223 ms 8 ae-1-54.bbr2.SanJose1.Level3.net (4.68.123.97) 2.322 ms ae-1-52.bbr2.SanJose1.Level3.net (4.68.123.33) 2.356 ms 2.361 ms 9 as-2-0.bbr2.LosAngeles1.Level3.net (4.68.128.15 13.796 ms 13.779 ms as-1-0.bbr1.LosAngeles1.Level3.net (209.247.9.113) 11.744 ms 10 ae-21-52.car1.LosAngeles1.Level3.net (4.68.102.44) 13.823 ms ae-11-53.car1.LosAngeles1.Level3.net (4.68.102.76) 11.755 ms ae-11-55.car1.LosAngeles1.Level3.net (4.68.102.140) 11.730 ms 11 * ge1-L3.dreamhost.com (4.78.192.66) 46.209 ms !A *
_____________________
Avatar: A temporary manifestation or aspect of a continuing entity.
|
|
Adam Zaius
Deus
Join date: 9 Jan 2004
Posts: 1,483
|
11-09-2005 13:29
From: Sapphire Bombay Basically it comes down to finger pointing and nobody from either company caring about it. Dreamhost says that LL's ISP is blocking. LL says that is not true. From the trace routes I see I tend to believe Dreamhost on that one (in fact it is this device: border5.fe5-23.linden-1.sfo.pnap.net (66.150.245.252)). Either way wouldn't you think LL would take a vested interest in helping us resolve this? I was going to say the opposite. I'd be betting on Dreamhost being the problem. I have dealt with LL's network people once before, about getting some evidence to throw at Level3 who were blocking traffic between SecondServer's two machines and about 30% of LL. I got the evidence from them, and Level3 removed the block after sufficient poking (this was probably the largest problem step), hasnt been another problem since (this was about 8 months ago, and LL's speed at helping us aquire evidence, and isolate the problem is admirable.). |
|
Adam Zaius
Deus
Join date: 9 Jan 2004
Posts: 1,483
|
11-09-2005 13:30
Also, your going to get yourself ignored if you stick only to traceroutes. Many providers block ICMP packets; which is going to give you false positives as to where the problem is occuring.
|
|
Sapphire Bombay
Avatar
Join date: 8 Oct 2003
Posts: 341
|
11-09-2005 13:39
From: Adam Zaius Also, your going to get yourself ignored if you stick only to traceroutes. Many providers block ICMP packets; which is going to give you false positives as to where the problem is occuring. But Internap is not blocking ICMP since I can successfully ping through to LL from my own PC.
_____________________
Avatar: A temporary manifestation or aspect of a continuing entity.
|
|
Krazzora Zaftig
Do you have my marbles?
Join date: 20 Aug 2005
Posts: 649
|
11-09-2005 13:40
Sapphire
I understand the one tracerouter from Mark Linden but where are you doing the other traceroute from? If from at home it will not work.
|
|
Sapphire Bombay
Avatar
Join date: 8 Oct 2003
Posts: 341
|
11-09-2005 13:47
slimy is a Dreamhost server. The one I'm hosted on. We get shell access.
_____________________
Avatar: A temporary manifestation or aspect of a continuing entity.
|
|
Krazzora Zaftig
Do you have my marbles?
Join date: 20 Aug 2005
Posts: 649
|
11-09-2005 13:49
From: Sapphire Bombay But Internap is not blocking ICMP since I can successfully ping through to LL from my own PC. Ok that's the problem right there. You are using two different paths. LL gets to Dreamhost by driving down let's say Interstate "A" You get to to LL by driving down Insterstate "B" Just cause you can get to LL on Interstate B doesn't mean there isn't a traffic accident causing tourble in Interstae A. What Mark is showing shows there IS trouble. P.S. Dreamhost might not be able to see this either. As thier side of Interstate A might be clear. |
|
Sapphire Bombay
Avatar
Join date: 8 Oct 2003
Posts: 341
|
11-09-2005 14:00
It doesn't matter which routes we take to get there. It is that border router sitting at Linden Lab that we all go through in the end. Notice my last device before I successfully get to to lindenlab.com is the same device that I have to go through when tracing from Dreamhost. border1.ge1-1-bbnet1.sfo002.pnap.net [63.251.63.1] The only difference is that my source address isn't Dreamhost in this case. So I get past that next device: border5.fe5-23.linden-1.sfo.pnap.net (66.150.245.252)
The filter is on this device.Home to Linden LabsFrom: someone $ tracert lindenlab.com
Tracing route to lindenlab.com [66.150.244.150]
over a maximum of 30 hops:
1 <10 ms <10 ms <10 ms NetgearLAN [192.168.0.1]
2 * 12 ms * acs-xx-xxx-xx-x.zoominternet.net [xx.xxx.xx.x] (intentionally obscured)
3 13 ms 14 ms 18 ms acs-xx-xx-xxx-x.zoominternet.net [xx.xxx.xx.xxx](intentionally obscured)
4 29 ms 32 ms 29 ms acs-72-23-2-97.zoominternet.net [72.23.2.97]
5 32 ms 28 ms 24 ms acs-72-23-2-29.zoominternet.net [72.23.2.29]
6 29 ms 25 ms 30 ms leg-208-30-204-161-ria.sprinthome.com [208.30.204.161]
7 52 ms 37 ms 48 ms 144.232.13.145
8 61 ms 62 ms 59 ms 144.232.9.156
9 80 ms 81 ms 80 ms 144.232.20.161
10 85 ms 86 ms 84 ms 144.232.15.142
11 110 ms 112 ms 108 ms 144.232.20.141
12 109 ms 120 ms 111 ms 144.232.20.113
13 111 ms 115 ms 105 ms 144.232.0.250
14 114 ms 109 ms 113 ms sl-internap-140-0.sprintlink.net [144.228.111.94]
15 111 ms 115 ms 107 ms border1.ge1-1-bbnet1.sfo002.pnap.net [63.251.63.1]
16 115 ms 114 ms 126 ms 66.150.244.150
_____________________
Avatar: A temporary manifestation or aspect of a continuing entity.
|
|
Krazzora Zaftig
Do you have my marbles?
Join date: 20 Aug 2005
Posts: 649
|
11-09-2005 14:03
ok now you are confusing data. Mark's post shows it failing on dreamhost's side and it failed due to administrative reasons which means dreamhost shut it down. You running a traceroute shows that trace..a one time attempt did not reach LL. A traceroute is a one shot attempt versus the hundreds you do each day. Just to play SL you do you thousands if not billions of those in a just a few hours.
EDIT: Ok better idea. If you are using your home computer and you say LL is blocking cause your traceroute shows it then you should NOT be able to connect to the game at all. No verifing username or password or any of that stuff. Try connecting and see what happens or run abotu 100 traceroutes in about 3 minutes total time span.
EDIT: I just did a tracert (windows version of traceroute) to both dreamhost and LL and had no issues whatsoever. The answers really do vary and can be different from second to second unless it is an administrative shutdown which LL is showing.
|
|
Huns Valen
Don't PM me here.
Join date: 3 May 2003
Posts: 2,749
|
11-09-2005 14:26
At this point, I'm really leaning towards getting out of the remote data business. If this had gone on a week it would be bad enough, but 20 or so days? Ouch. I'm sorry, but I am just not prepared to deal with this level of risk. Hopefully in the future I will find reason to have more confidence.
|
|
blaze Spinnaker
1/2 Serious
Join date: 12 Aug 2004
Posts: 5,898
|
11-09-2005 14:33
LindenLabs is the only one getting: 11 * ge1-L3.dreamhost.com (4.78.192.66) 46.209 ms !A * Everyone else (you, me, tracroute.org) gets 9 FE1-MZ.dreamhost.com (216.193.192.50) 5.080 ms 4.680 ms 4.603 ms 10 slimy.dreamhost.com (205.196.208.1 6.352 ms 4.839 ms 7.460 ms Clearly, someone is administratively filtering LindenLab specifically from tracerouting to Dreamhost. Mark .. do you know of any dedicated hosting services at your colo that you can recommend? Should we just call internap? |
|
Krazzora Zaftig
Do you have my marbles?
Join date: 20 Aug 2005
Posts: 649
|
11-09-2005 14:46
Blaze
Intermap won't have the answer. Dreamhost is being provided Level 3. The administrative shutdown is therefore being done by one of those two and one of those two companies will be the only ones that have answers.
|
|
blaze Spinnaker
1/2 Serious
Join date: 12 Aug 2004
Posts: 5,898
|
11-09-2005 14:48
I was asking for dedicated hosting at internap.
I'm sure internap would have the answer.
|
|
Krazzora Zaftig
Do you have my marbles?
Join date: 20 Aug 2005
Posts: 649
|
11-09-2005 14:53
Sorry for the mix up on the hosting question but trust me on Internap. They would not know about the administrative shutdown until told by LL or another client. I've had to deal with one where my ISP was blocked by AOL for 2 weeks. Duing that time my ISP didn't know about for the first day or two until they got hundreds of calls asking why AOL members could email them but no replies were getting back to AOL customers.
Mark is doing the right thing and can do nothing more. He is waiting to hear back from Dreamhost last we heard and so right now it is Dreamhost that is not giving him the time of day to answer him.
|
|
blaze Spinnaker
1/2 Serious
Join date: 12 Aug 2004
Posts: 5,898
|
11-09-2005 15:01
After 20 days, I'd start calling main switchboards and get through that way rather than following normal escalation procedures.
He already has first names, I'm sure if he says "Brian in networks" he might get through.
|
|
Krazzora Zaftig
Do you have my marbles?
Join date: 20 Aug 2005
Posts: 649
|
11-09-2005 15:04
True concidering it has been 20 days they might escalate it faster.....but one customer out of how man thousands of customers?
|
|
Adam Zaius
Deus
Join date: 9 Jan 2004
Posts: 1,483
|
11-09-2005 15:37
ZOMG! Here's why using traceroutes as your executive result. I just ran a trace from slscripts.com (one of the two SS.net machines). Here's the results. Note: I can happily connect to SL's email & XMLRPC side of things from this machine, just fine. From: someone slscripts:~# traceroute lindenlab.com
traceroute: Warning: lindenlab.com has multiple addresses; using 66.150.244.150
traceroute to lindenlab.com (66.150.244.150), 30 hops max, 38 byte packets
1 10.255.255.253 (10.255.255.253) 4.168 ms 9.635 ms 2.154 ms
2 v998.gw-core-a.nyc.schlund.net (217.160.229.125) 0.292 ms 0.245 ms 0.332 ms
3 ge-120.gw-backbone-a.nyc.schlund.net (217.160.229.97) 0.315 ms 0.309 ms 0.310 ms
4 144.232.228.5 (144.232.228.5) 0.282 ms 0.342 ms 0.183 ms
5 sl-bb24-nyc-15-2.sprintlink.net (144.232.13.21) 3.340 ms 0.787 ms 0.566 ms
6 sl-bb25-nyc-10-0.sprintlink.net (144.232.13.182) 0.714 ms 53.123 ms 1.187 ms
7 sl-bb24-chi-2-0.sprintlink.net (144.232.9.156) 24.370 ms 24.973 ms 23.943 ms
8 sl-bb20-che-2-0.sprintlink.net (144.232.20.161) 43.362 ms 43.123 ms 43.245 ms
9 sl-bb21-che-15-0.sprintlink.net (144.232.15.142) 43.223 ms 43.303 ms 43.303 ms
10 sl-bb22-stk-6-0.sprintlink.net (144.232.20.141) 69.444 ms 69.525 ms 69.389 ms
11 sl-bb23-sj-10-0.sprintlink.net (144.232.20.113) 74.076 ms 183.204 ms 85.566 ms
12 sl-gw19-sj-15-0.sprintlink.net (144.232.0.250) 191.649 ms 192.434 ms 247.916 ms
13 sl-internap-140-0.sprintlink.net (144.228.111.94) 74.107 ms 74.300 ms 74.072 ms
14 border1.ge1-1-bbnet1.sfo002.pnap.net (63.251.63.1) 74.159 ms 74.066 ms border1.ge2-1-bbnet2.sfo002.pnap.net (63.251.63.65) 74.162 ms
15 * * *
16 * * *
17 * * *
Note, the same router. It's killing ICMP packets. Hence, your NOT GETTING VALID DATA. From: someone slscripts:~# ping lindenlab.com
PING lindenlab.com (66.150.244.150): 56 data bytes
64 bytes from 66.150.244.150: icmp_seq=0 ttl=50 time=74.1 ms
64 bytes from 66.150.244.150: icmp_seq=1 ttl=50 time=74.1 ms
--- lindenlab.com ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 74.1/74.1/74.1 ms
From: someone slscripts:~# ping sim500.agni.lindenlab.com
PING sim500.agni.lindenlab.com (69.25.104.186): 56 data bytes
64 bytes from 69.25.104.186: icmp_seq=0 ttl=50 time=74.1 ms
64 bytes from 69.25.104.186: icmp_seq=1 ttl=50 time=74.1 ms
--- sim500.agni.lindenlab.com ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 74.1/74.1/74.1 ms
Edit: For completenesses sake From: someone slscripts:~# telnet data.agni.lindenlab.com 25
Trying 66.150.244.192...
Connected to data.agni.lindenlab.com.
Escape character is '^]'.
220 data.agni.lindenlab.com ESMTP Postfix
HELO
501 Syntax: HELO hostname
^]
|
|
Eggy Lippmann
Wiktator
Join date: 1 May 2003
Posts: 7,939
|
11-09-2005 15:41
What about the "!A" bit in Mark's post?
|
|
Krazzora Zaftig
Do you have my marbles?
Join date: 20 Aug 2005
Posts: 649
|
11-09-2005 15:50
From: Eggy Lippmann What about the "!A" bit in Mark's post? That is what I am trying to get across. the !A is next to a dreamhost router. the "!A" itself as he states means they are getting rejected at that point cause LL has been administrativly shut down from getting information past that router. Problem is only LL and/or anyone else effected by this will get it. WE as you you and me Eggy are OT affected by this as these are usually done by IP address range or Domain name. So it could be as deep of a ban as Internap (Assuming that is who LL rents thier space through) or just LL. Anyone of us contacting Dreamhost or LL will be able to get to the server via pig, traceroute, or connecting to thier services (all valid ways of showing connection or levels of connection) think of this as literally the US highways and interstates and righ t now the city of LindeLabs is not allowed access to the City of DreamHost. Now we need to find out if it is just the city or the state (Level 3) that DreamHost is in that is banning LL. Incidently also think of sending as being the northbound side of a road and the recieving as the southbound. |
|
blaze Spinnaker
1/2 Serious
Join date: 12 Aug 2004
Posts: 5,898
|
11-09-2005 16:47
Yah, I went over this with Mark, Adam. Read back in the thread.
Try traceroute -I
internap is filtering udp packets, but not ICMP .. -I will do ICMP.
Confuses the issue a bit, but not by much.
|
|
Mark Linden
Funky Linden Monkey
Join date: 20 Nov 2002
Posts: 179
|
11-09-2005 17:11
All of the folks attempting to traceroute to us should know that by default, unix traceroute using UDP packets, not ICMP.
We block UDP that we don't need; this is why it appears that the router at pnap.net is the last hop you see; the hop after it is one of our servers (which ever one you are tracerouting).
Try traceroute -I instead; it will get through if you are not coming from Dreamhost.
Also, note that if Dreamhost's gateway router has an IP ACL installed in the way that I suspect, it would appear that traceroutes from their network to us would fail at the last hop (since the reply to the traceroute would be from an IP on their block list).
Does that make sense?
M
|
