Verizon Fios – The Ugly Side.

I’ve talked about how great Verizon Fios is previously. And at that time I noted a few downsides that hurt the service quality but still noted the ISP was the best I’d used. Today I’m going to rant a little longer about the ugly side of FiOS, mainly because I have just gone through/am still going through an excruciating experience[1]. That said, I still recommend FiOS[2].

On Thursday, 10/16/08 I tried to log onto the internet. No connection. Between 10/16-10/17 I would make seven major phone calls to Verizon in an attempt to resolve this issuing totaling approximately 250 minutes (over 4 hours). Verizon for an unknown reason[3] disconnected my FiOS service[4]. Okay, no big deal – reestablish the service[5]. They create an entirely new account and inform me at the same time that I can no longer have my less-expensive 5 Mbps/2 Mbps subscription but must upgrade to a 10 Mbps/2 Mbps. Again, not a huge deal – $10/mo. – but doesn’t make an unhappy customer happy when you screw up and then demand he pay a higher price for the service.

The installation was supposed to be completed by the end of that day. At the end of the day I called back, ohh – somebody screwed up the order. I would talk to approximately three people between that call and calls the following day attempting to get the order straightened out. They placed at least two new orders and finally one of them kicked through. In all, it was around 48 hours without internet access.

Then the internet access finally returned. Great! I’m a happy man…Till its slower than DSL. I run some speedtests. Yup, certainly slower than DSL – and ping tests against Google/Yahoo’s servers revealed that occasional timeouts where occurring and response times were slow (over 100 ms typical). More phone calls to Verizon[6]. Yes, my service is now working. No, the service is very slow. How do I know? Well, I used a bandwidth test. Run through the normal diagnostics – resetting wireless router, plugging directly into FiOS connection, resetting the FiOS box outside my house – sending the issue over to a tech. to look at. The pings drop down – Google/Yahoo now in the 10 ms range, but the speed is still horrific. Maybe its the bandwidth tester – well, I trust Speakeasy’s test but, sure, let’s try Verizon’s. Verizon shows even more dismal speeds. Ahh, forget Verizon’s speed test let’s try speedtest.net. Now if I go to the server in Newark I get 10 Mbps, but almost everywhere else its slow as anything. Will I run the FiOS settings optimizer[7]? Sure. Still nothing. I’m tired, I’m going in circles. I talk to a supervisor. Same thing. They want to send me a new router. Sure, you can do that, but since I’m plugging directly into the connection and bypassing the router it’s not the router. Okay, we’ll have a technician look at it. Great – never hear from them again.

The connection has slowly improved. Dropped packets have decreased, but overall the network is slow. Streaming video doesn’t work well – unless you happen to hit 1% of the servers that seem to be operating on Verizon’s network at high speeds.

So, that is my situation as of today. Two days without internet – now extremely slow internet. Calling into technical support is an exercise in frustration. I can’t speak to someone who understands the problem is a technical issue on Verizon’s side. My guess? Either my circuit settings where entered wrong or they have a bad router somewhere along the line. 99% of my traffic is going through the bad router – if I happen to hit a server that takes another route (doesn’t use that router) then I get good speeds.

Anyone else having this problem? The phone support from Verizon has improved in that I don’t spend 15 minutes dialing through automated voice support – but the help desk support doesn’t have a clear escalation path beyond themselves and so I am at an ends. Vague promises of a “network tech.” looking into it and no followup leaves me with little hope of a long-term solution.

UPDATE:

Spoke to a technician at Verizon who knew what I was talking about and again submitted a ticket to the network tech. guys. His name was Dan…If any Verizon people do read this, he needs a promotion – his networking knowledge is far beyond most of his peers. In any case, he agrees the issue is one with the route being taken to reach the final website. To give everyone an idea here is a tracert to google.com (note the length of the route, its extremely, ridiculously long):

1 1 ms <1 ms <1 ms Wireless_Broadband_Router.home [192.168.1.1]
2 4 ms 4 ms 4 ms L100.FTTP-02.PHLAPA.verizon-gni.net [68.238.240.1]
3 7 ms 6 ms 7 ms G1-0-881.LCR-08.PHLAPA.verizon-gni.net [130.81.110.108]
4 7 ms 6 ms 7 ms P14-2.LCR-02.STTLWA.verizon-gni.net [130.81.29.6]
5 7 ms * 7 ms 0.so-6-0-0.XL2.PHL6.ALTER.NET [152.63.3.81]
6 13 ms 12 ms 11 ms 0.so-6-0-0.XL4.IAD8.ALTER.NET [152.63.0.130]
7 13 ms 11 ms 12 ms 0.ge-3-3-0.BR2.IAD8.ALTER.NET [152.63.40.229]
8 22 ms 49 ms 99 ms 204.255.169.2
9 88 ms 86 ms 87 ms tbr1.wswdc.ip.att.net [12.123.8.106]
10 580 ms 576 ms 724 ms cr1.wswdc.ip.att.net [12.122.16.89]
11 580 ms 484 ms 96 ms cr2.phlpa.ip.att.net [12.122.4.53]
12 115 ms 97 ms 96 ms cr2.cl2oh.ip.att.net [12.122.2.209]
13 87 ms 87 ms 86 ms cr1.cl2oh.ip.att.net [12.122.2.125]
14 88 ms 87 ms 86 ms cr1.cgcil.ip.att.net [12.122.2.205]
15 99 ms 99 ms 99 ms cr1.st6wa.ip.att.net [12.122.31.162]
16 88 ms 86 ms 87 ms tbr1.st6wa.ip.att.net [12.122.23.130]
17 87 ms 86 ms 87 ms gar8.st6wa.ip.att.net [12.122.111.17]
18 98 ms 96 ms 117 ms 12.89.209.14
19 111 ms 96 ms 97 ms 209.85.249.32
20 103 ms 98 ms 89 ms 66.249.95.208
21 91 ms 92 ms 91 ms 72.14.233.37
22 92 ms 92 ms 104 ms 216.239.48.143
23 93 ms 106 ms 117 ms 209.85.251.125
24 102 ms 94 ms 104 ms 74.125.31.2
25 105 ms 104 ms 104 ms cg-in-f99.google.com [209.85.171.99]

Unfortunately 99% of the sites I visit appear to be routing in this same, obtuse method. 1% take a different router and operate at blazing fast speeds.

  1. [1]I rant in part about these types of situations because the bureaucracy in these large companies makes it almost impossible to get help. Oftentimes a blog posting is the most effective way to get real help.
  2. [2]I recommend more the technology than the service. At this juncture there is no alternative provider with the same technology available in this geographic region. On a similar note, the use of traditional DSL through alternative providers does not actually circumvent utilizing Verizon as these providers operate over Verizon’s lines. Satellite/Cable are true alternatives and WiMax (Sprint’s Xohm) when it becomes available will be another real alternative
  3. [3]Not only unknown to me but also unknown to them. They did not know why I was disconnected, just that they had done it.
  4. [4]Second time this has happened since establishing service with them.
  5. [5]Unfortunately it takes hours for this to occur. As a Network Engineer I am still unable to comprehend what could possibly take so long in the assignment of an IP address when all the physical wiring actually exists and has for some time.
  6. [6]Unfortunately my cell records have not updated to show me the length of those calls, so I cannot report their exact duration – but it was several and the time was significant.
  7. [7]Glorified IE/TCP optimizer.