Is all of Vonage completely down?

Posts: 448 · Posted: Wed Aug 03, 2005 12:42 pm Post subject: Re: "Oh well" again

rickster wrote:

I love Vonage for my home.... but I would not THINK about using them for a business line. That would be suicide for a business. That is why I have no sympathy for your outage.

Ditto. We ported the home number to Vonage and are saving a lot of money... but the service seems to be degrading. (more and more "all circuits are busy" messages to inbound callers, for instance) Probably has to do with the sheer volume of new accounts and the inability of "partner carriers" to keep up.

At work, we ported some numbers but left the primary number on POTS. Worst case scenario is we're down to one (highly reliable) line.... and it's saving us a small fortune on long distance.

Posts: 448 · Posted: Wed Aug 03, 2005 12:49 pm Post subject: Re: Silly

rickster wrote:

Grizzley,
You're right, I do assume that Vonage is not taking the steps I have outlined. If they are then their IT must be incompenent.

Maybe not incompetent, but it appears they have not sufficiently implemented redundancy. We must assume there are one or more single points of failure, because it would be somewhat improbable that the numerous failures we've seen in the past 6-12 months were bought on by a simultaneous failure of all redundant elements.

Redundancy costs money, and considering what must be a fairly thin margin per-account, my guess is they aren't willing to spend that kind of money yet for the redundancy needed to prevent these kinds of system-wide outages.

Posts: 3 · New Forum Member Joined: Aug 03, 2005 Posts: 3

Whatever patches Vonage installed recently or whatever marketing program they launched appears to have found a business that looks like it can't manage projects. Since we don't know the facts, we will just say thats' the view from outside.

In the past two weeks, voice mail has come to a standstill in the morning and afternoon and now the outbound calling system has failed.

Voice Mail: I was told by support these were peak time issues and there was nothing they could do. I have to live with a 30 second delay in getting to the prompt or double prmopts if I hang up and try again.

Service Down 8/3/05: All outbound calling seems to have failed. Ring no answer, fast busy, or garbled ring. Support says everyone is working on it. Good for them.

Is this poor software release control?

Is this poor capacity planing?

Is this just what it is - a company unable to live up to the promise of its marketing message.

Calling support - It takes 5 to 10 steps to get to where you need to go - This is what happens when complaints arise about off-chore support. They hand the customer the Script Tree and make them figure out the problem. Now that we have to use the front end of the script tree, its clear why we are frustrated with support. Not all the problems are listed in the prompts and the support staff may have never used Vonage.

Lesson here: Early adopter tend to be smart and take risks, don't hand them to the Late Majority who are usually unprepared to program a VCR.

Posts: 4 · New Forum Member Joined: Aug 03, 2005 Posts: 4

the least they can do is email an alert to those affected by the outage. What good is it to have an alert on the webpage if you cant access the webpage because it is down.

I agree with Rebus, apparently they have not implemented sufficient redundancy, one thing is to lose a couple of calls or maybe a couple of states coverage, but its another thing entirely when the entire Vonage system goes down including the companies own website.

Posts: 3 · New Forum Member Joined: Aug 03, 2005 Posts: 3

[quote="rebus"]

rickster wrote:

Grizzley,
You're right, I do assume that Vonage is not taking the steps I have outlined. If they are then their IT must be incompenent.

Perhaps this is less of a redundancy problem and more of a procedural one. recently Vonage announced they were going to begin delivering business grade services that include features like v/pbx. Is it possible these guys don't know enough about version control, load testing and QC? If you drop an upgrade on both your primary and backup systems, and the upgrade has a problem, what do you have? No redundancy.

Posts: 34 · Vonage Forum Junior Joined: Apr 05, 2005 Posts: 34

rickster wrote:

Grizzley,

I know you mean well, but this if they want people to accept a paradigm shift in phone service, they have to improve the following:

* "Off box" Monitoring. They should have points of presence doing off site monitoring of their protocols for availabilty at strategic points all over the internet and their internal network to allow them to immediately pinpoint network issues.

* Server monitoring. All their servers should be instrumented to constantly monitor key metrics and take appropriate actions based on practive policies, raise an alert if anything looks like it is *about* to go wrong, and scream like crazy if something does go rwrong.

* Invest in state of the art, robust root cause analysis software to correlate the above and quickly pinpoint issues.

* Redundancy and failover. Their entire backend infrastructure should be redundant - routers, switches, servers, power, etc, and failover should be automatic, transparent, and raise the appropriate alerts during and after the fact. This includes a certain level of geographic redundancy.

* Adequate parallel testing before applying patches, upgrades, etc. I'm talking about software created internally.

* At the very least, what I have described above should apply to their Web site. In 2005, the software and hardware required are well known and are in fact commodity items. It borders on negligence to have a Web site go down this often. Who is running IT for these guys?

That is not a realistic business model. "Build it and they will come" only works in the movies.

They have to oversell and put their revenue into advertising, not infrastructure if they want to survive.

I saw the exact same thing in the Internet boom in the 90's. There were companies that had fantastic, reliable infrastructure and they went broke. Then there were other companies that started with an ad campaign and barely built the connections as they went. They are the ones that are now the big players.

It ****, but it is unfortunately what society has brought upon itself. The stock market boom of the 80's turned the dorky stock broker into the multi-millionaire. More people followed stocks, companies required higher profits to keep investors happy, the result was the quality loss of the 90's and today.

I disagree with the way Vonnage runs its business. I choose not to run my business that way, but it does reflect in the slow growth of my company in the market. I am growing much slower than other competitors who sell more than deliver but I don't care, I can pay my bills and stash something away for retirement. That type of mentality doesn't sit well with stock holders.

Doug

Posts: 3 · New Forum Member Joined: Aug 03, 2005 Posts: 3

SBMongoos wrote:

That's hilarious.

No service in Denver either. In our outbound. Nothing works.

By the way. Why is it my number shows up twice on someone elses CID instead of my name and my number. Same for my girlfriend who uses Vonage.

The way the system is setup using switches owned by other companies, one part of the CID is not sent - your name, only your phone number. I am not sure if this is a protocol issue or design problem. It is on the list of things Vonage would like to address.

Posts: 3198

cid info isn't sent, it's looked up by the receiving telco. Vonage doesn't enter your info in the cnam or ldap databases (one or the other is the primary cid lookup database, maybe both, I don't remember). so the telco is forced to handle it in whatever way it does. usually Vonage to Vonage calls show the number twice. sometimes the name comes up as Vonage holdings or something similar, which can be changed to blank if you call customer service. other telco's interpret blank as "indiana call" or whatever state it originates from. here is a good explanation:
http://www.dslreports.com/forum/remark,11327439

vonagraine wrote:

SBMongoos wrote:

By the way. Why is it my number shows up twice on someone elses CID instead of my name and my number. Same for my girlfriend who uses Vonage.

The way the system is setup using switches owned by other companies, one part of the CID is not sent - your name, only your phone number. I am not sure if this is a protocol issue or design problem. It is on the list of things Vonage would like to address.

Posts: 148 · Vonage Forum Senior Joined: Feb 28, 2005 Posts: 148

Yep, cannot make outbound local calls here (get the "all circuits are busy" message), but I can make outbound long distance calls just fine. Inbound local calls work fine here (tested this with my cell phone. Very strange.

I should add, I live near Champaign, IL.

Posts: 2 · New Forum Member Joined: May 12, 2005 Posts: 2

Although I got an "All circuits are busy" message when I first head about this issue, I did notice there was constant Internet traffic showing on my Linksys WRT54GP2. After shutting everything on the network down to be sure that none of the computers were causing the traffic, I enabled logging and saw that the traffic was coming from three Vonage IP addresses (216.115.21.26, 216.115.31.140 and 64.192.218.178). Incoming calls are working. It just seems odd there's that much noticeable traffic on the router.