The most common SNAFUs when planning and implementing an open source VoIP PBX rollout and how to avoid them

This is a general overview of some of the more common mile high mistakes people make when doing an open source VoIP PBX project.  I won't be going into details about hardware dimensioning or software configuration.  I will just be providing a general overview of some of the system level common sense mistakes people make in terms of a typical consultant/client type of project.

This is a subject that is very near and dear to my heart.  I visit several different Asterisk/FreePBX type message boards regularly and often cringe at some of the things people are doing.  It's none of my business other than the fact it affects the reputation of our industry as a whole so in that respect it IS my business.  The great thing about using Asterisk for a VoIP PBX is its incredible flexibility and scalability.  Nothing even comes close in my humble opinion.  However, this strength is also its weakness.  It seems familiar and relatively easy for people comfortable with Linux software and standard computer hardware.  With the various pre-packaged distributions out there it would seem easy even if you know little if anything about Linux.  Having said that, because there are so many different ways of doing it there are many ways of doing it wrong.  A lot of computer and network consultants getting into VoIP continue to do things the way they have successfully done it in IT for years.  With VoIP it's a whole new minefield out there.  I read about people stepping on these mines daily.

I'll be giving some of my opinions on what I see as common mistakes and provide some simple tips to avoid them.  As always, these are just my opinions and may not be the best way for everyone in every situation.  It's just what I do for better or for worse.  I will be talking strictly business production VoIP PBX systems.  These are systems that will be used in serious business environments where a properly working and highly reliable phone system is critical to the day to day operation of the business.

Mistake #1, cheap hardware
If you want to build a good house you have to have a good foundation.  If you want to build a good software PBX you absolutely positively MUST have good hardware.  Not a week goes by that I do not come across someone somewhere who is planning to put in a PBX using cheap hardware.  Some justify it by having spare hardware so if and when the cheap stuff fails they can quickly replace it with the same cheap stuff.  Won't that impress the client!  Worse yet they have already put it in and the customer is freaking out over problems that are negatively affecting their business. The consultant is now desperately trying to find quick fix solutions without having to pour more money in. The price expectations have already been set and it's very hard explaining to the client that they are now going to have to fork out more money.  More often than not these consultants are doomed to failure because they didn't use the right hardware for the job in the first place.  In my opinion, if you feel that you have to cut corners to get the price down and get the business you should ask yourself if you really want that business.  In my experience, customers who place the greatest emphasis on price are not very good clients.  Sometimes it's better to just walk away and let your competitors get the headaches.  Better yet, the low bidder will screw up so badly that the customer will come back begging for your help.

So what is good hardware?  For the server I only use and recommend commercial grade equipment.  I personally prefer to use Supermicro hardware.  Commercial grade hardware is ALL they do.  It's also preferable to use a custom enclosure with your own company labeling on it to appear even more unique.  If you prefer some of the more recognizable name brand equipment stick with their mid range server equipment even if it seems like overkill that is overpriced.  Stay away from their low end server equipment which can often be just as bad if not worse quality than standard desktop computer hardware.  I don't like to use the big name brand equipment myself.  The onsite replacement warranties and 800 number support isn't of much use for a VoIP PBX other than to provide a false sense of security in my opinion.  If a hard drive fails they may be able to send someone out to replace it but someone still has to do the bare metal software install if you’re not using RAID (which you should).  If a motherboard fails you should ask yourself if the replacement will fail as well. Whoever answers that 800 number call is probably not going to be able to help with Linux software issues.  Also, using big name brand hardware in standard enclosures with their name on it tends to make the customers see your services as more of a commodity.  You should always strive to be seen as a specialist with the unique knowledge, experience, and hardware to get the job done right.  If you’re seen as just another IT guy selling Dell hardware then they will constantly be asking themselves why they shouldn't just cut out the middleman and get their computer geek friend to do it for nothing.  Again, you really don't want those sorts of customers anyways.  Trust me!

As far as Telco cards, things are constantly changing.  With new competitors entering the market and cheap knock off equipment coming out of China it's more confusing than ever.  Hardware warranties do mean something with Telco cards if not for anything else than the fact they are willing to stand behind their product.  So far the only cards I have seen coming out of China are based on the same poor design as the original Digium TDM400 card or worse yet the infamous X100P card.  Stay away from these cards.  The PCI interface is poorly suited to telephony for reasons I won't bother going into.  Suffice it to say, these cards have managed to single handedly give open source VoIP PBXs a very bad reputation right when they are struggling for commercial acceptance.  Again these are just my opinions.  I realize there are many people who have successfully rolled out systems using this hardware but probably just as many people have had problems.  Why roll the dice in the first place when you’re trying to build a good reputation.  I personally have only used Sangoma hardware up until now.  Their cards mostly just work and their support is excellent.  It sounds like the Rhino cards are now working well so I will probably consider them for more cost sensitive projects.  Both have offered at least a 5 year warranty up until now.  Sangoma recently announced they now offer a lifetime warranty.

Last but certainly not least are the SIP phones. The phone is the single most critical piece of hardware in terms of the customer experience and a good quality phone makes all the difference.  You can't go wrong with Aastra or Polycom in my opinion.  I prefer Aastra for the combination of great quality, great features, good price, and relative ease of setup.

Mistake #2, cheap support
VoIP Telephony is not easy to do right.  I've been in IT for quite some time and it took me at least 2 years of experimenting with Asterisk and Telco hardware to get to the point where I felt comfortable offering my services.  Why sell yourself short?   In my opinion the service and support required to plan, install and maintain a high quality high reliability VoIP PBX phone system is worth much more than in IT.  Sure, just about anyone can take one of the off-the-shelf distributions out there and set up a phone system that will probably work ok.  That is not your target market anyways.  But when things go wrong these people are the ones on the message boards desperately looking for someone to help them with their problems.  Meanwhile their business is losing money.  I've been there and it's not fun.   No matter how good the pre-packaged distributions get, Telco will continue to be a very specialized area requiring specialized knowledge.  You should charge accordingly for that knowledge keeping in mind that a phone system is usually the single most important piece of technology a business needs.  The traditional Telco vendors charge rates that are usually much higher than IT rates and you should do the same.  Don't get me wrong, I don't do this because I think I can get away with it.  I am doing this because the expectation for high quality service and support for Telco equipment is higher than in IT and I fully intend to meet those expectations.

Mistake #3, using SIP trunks as a replacement for traditional TDM trunks
A SIP trunk is basically x number of virtual IP phone lines between you and some SIP trunk provider that ultimately terminates the lines to the traditional public switched telephony network.  The problem is that IP traffic over the internet is a best effort kind of thing.  There is no end to end quality of service guarantee so there is no guarantee of call quality and reliability.  There are several variables that can improve the statistical likelihood of good quality and reliability but NEVER a guarantee like there is with TDM trunks (plain old analog lines or digital lines).  It does not matter how good the SIP trunk provider’s reputation is or how close (in terms of ping time and hops) the provider is to you or how good their PSTN connection is.  As long as your packets have to compete with other public internet packets as they travel through one or more Internet service providers networks there are no guarantees.  SIP trunks should only be viewed as a more cost effective way to increase the number of phone lines available and/or a way of reducing long distance charges. 

When choosing a SIP trunk provider you should look for someone that terminates to the PSTN relatively close to you.  Ask for the IP address of their SIP server or some other ping able server in the same facility.  Ideally you should see a minimal hop count (say less than 7) and a ping time of less than 50ms.  Anything more is risky for consistent quality and reliability.  Also ensure that they terminate to the PSTN in the same facility.  Many discount SIP providers outsource PSTN connectivity to a wholesaler that may be in another facility hundreds or thousands of miles away that would again have to travel from their SIP servers across the internet.  You won't see this if you’re pinging their SIP servers.  If they are not willing to readily provide this information then you should take that as a red flag.

Mistake #4, over promising and under testing features
So the client won't commit unless it can make toast?  Sure no problem, that's included in the price.  That is what makes Asterisk so great.  Just about anything is possible.....but......will it meet the client’s expectations?  There are so many things Asterisk can do well.  Unfortunately, there are many things Asterisk cannot do well or as well as closed source or traditional Telco equipment.  For instance, if a client expects shared line appearance (Traditional KSU behavior) I will not try selling them an open source system.  The SLA implementation on Asterisk 1.4 is not production ready and probably never will be.  Even if you were to get it working reasonably well it cripples the system such that you can't do many of the other things that Asterisk/Freepbx does so well.  Customers who expect KSU type shared line appearance behavior are simply NOT a good fit for Asterisk/Freepbx.  Some people try selling the customer on call parking which may or may not fly with them.  Don't do it without first demonstrating it to the customer and getting their buy in.  Also, I don't recommend selling any feature without first extensively testing it yourself.  I only promise the tried and true features of Asterisk/Freepbx which are more than enough to compete with anything else out there.  If the client wants some of the more esoteric features such as CRM integration or any one of hundreds of other things that are possible I generally won't provide a quote until I have fully tested and verified it.  At the very least I would want to have a high degree of confidence that it will meet the client’s expectations.  That does not mean a couple posts on some message board or the VoIP wiki from a couple guys who say they got it working in their house and had it tested by their teenagers.

Mistake #5, updating the software without good reason
ln the Microsoft or even the Redhat world it may seem like a no brainer to update server software just because the update is available and may fix some vulnerability or make something work better or maybe fix some problem you didn't even know about.  In the PBX world I tend to think of an Asterisk/Freepbx server as more of an embedded network appliance.  It probably shouldn't be touched unless you have a specific problem you are trying to fix. Even so you should only patch/update for the problem and not just general updates.  I can't count the number of times I have read someone posting about having an Asterisk PBX server running nicely only to bring the business to it's knees after a routine update or upgrading to the latest and greatest version.   Sometimes it's done just because the new version supposedly has a lot of cool new features that the person would like to play with or some other low priority reason.  My recommendation is to resist this typical IT type behavior as much as possible and save that sort of thing for experimentation on a test system. If it's not broke don't fix it.  A software PBX may in theory be considered an application server but it shouldn't necessarily be treated like any typical IT application server.  A phone system is usually a very critical part of a business that is not tolerant of even a few minutes of down time or flaky behavior much less a few hours or days.  IT people are usually more familiar with applications that are more tolerant to outages and less than 100% perfect operation such as email, file serving etc.  Yea people may complain about those things too but if they can't make phone calls it's usually a whole other level of distress and urgency that can be a whole lot more stressful to experience.  I've been there, done that and got the grey hairs to prove it.  The potential for these problems is not something to be taken lightly in my humble opinion. 


That's it for now.  As I think of more things I will probably revise this document.