Hostway (and web hosting in general) sucks
Update 10/31: I just want to say that Hostway isn't as bad as they may sound in this post; they've been helpful with most of the issues that I have had and I don't hate the fact that my employer is hosted with them. I just think Hostway might be a bit too big to really be the best thing for all their customers. They're a very large hosting company and it's hard to be everything to everyone. All things considered, though, I stand by the things I said in this post.
When I started working at my new job, being the only developer naturally meant that I would inherit all the "oh my gosh the server isn't working" responsibilities. I didn't mind that too much at first since I had some experience modifying my own server (which hosts this site). What I did not realize, however, is that the company my employer uses for web hosting, Hostway, is very different from the one I'm accustomed to.
My employer currently uses the "Advantage" server plan for hosting their sites, 3 of which get a lot of traffic. This is a "managed" setup meaning that we can upload and modify files on the server but we do not have root access. Before I started working, Hostway had just transferred the sites to a new Advantage server because the previous one had unexplained hard drive problems. Not long after, the new server started having performance issues, so we purchased more RAM, taking the server from 1 GB to 3 GB. You would think that 3 GB could handle the load, but it couldn't. So I had to figure out the next step.
These sites are not at all optimized, but more importantly, Advantage servers are not optimized either. PHP & Perl simply run as CGI without FastCGI and with PHP running as CGI, it's not possible to use an optimizer like APC. When I asked if FastCGI could be installed on our Advantage server (which apparently can double performance on its own), I was told that it presented a security risk and that we would have to go to the "Truflex" dedicated servers to be able to do that ourselves. When I found the sites insufferably slow last week, I decided enough was enough and started looking at options.
I was intent on getting an optimized setup because there's a lot of things I have yet to build on these sites and I want to be certain that the server can handle more load. I had come up with 2 options:
- Drop Hostway and sign up with a different company that offers optimized server setups already
- Sign up for Hostway's "Truflex" dedicated servers and set it all up myself
I opted for the first option but that didn't fly; it was safer to stick with the company we knew rather than jumping head-first into another. I didn't want to stay with Hostway since we had already had problems with them and their Advantage support had been useless in the past but I couldn't argue with such a cautious evaluation of the situation. So, I decided to order 3 dedicated servers that would each host one of our high-traffic sites and migrate everything over myself. When I ordered the servers I asked for CentOS since it's the one I have some experience with.
Today was the day that I started the migration. Looking back, I probably shouldn't have thought that I could do it. While transferring the database and all the files was easy, I could not get PHP/Perl/etc. installed the way I wanted it. I was especially surprised to discover that on this new server, PHP was installed as CLI (Command Line Interface). While I was able to test some scripts and even connect to the database, I was very dissappointed to find that none of my PHP scripts could read or write files. I also could not for the life of me figure out why Perl files and CGI files sitting in cgi-bin/ were not working.
I tried a lot of things. I actually spent the whole day trying to get it to work. I installed FastCGI, mod_fcgid, a bunch of packages, etc. I tried a number of changes to httpd.conf and nothing worked. I went through countless pages online looking for a tutorial that would work, but everything I tried failed. It was depressing. The day went by and I still couldn't get the server working. All I wanted to do was configure FastCGI and use it to run Perl and PHP with sufficient permissions to read and write, and it wasn't happening. Hostway's support staff was totally unhelpful too, since the guy I talked to didn't actually know anything about server software (he was just a "networking" guy).
Now you might say, "oh well, things didn't work out, just get a refund," but that would bring up a huge reason why I think Hostway sucks; everyone gets charged a $99 setup fee for each server. As of today, I had spent $297 getting servers that still don't work. Plus, there was all my time lost during the day and all the time that is going to be lost if and when I can finally fix things. What I just can't understand, however, is this: why don't webhosts offer servers that are already optimized? You know, like, PHP 5 runs as FastCGI, with APC installed, and all the security is set up for PHP/Perl/CGI. These are very reasonable requests, so much so that I've gotten to thinking that these thing come standard with any server. Most web hosts don't do this stuff, though, and Hostway sure doesn't. I really don't know how people tolerate the whole server-software-config game.
So I'm stuck, because Hostway charges $150 by the hour for support to dedicated plan customers and they wouldn't even know what to do. The solution right now is to give up on these new servers and go back to the Advantage servers which at least work already. Or move to another host, which is what I said to do in the first place. Still, I'm not going to forget how frustrating today has been or how stupid it is that they offer two extreme spectrums of hosting without any correlation between them. I just wish I had a way of making these servers work. Anyone there know anything about CentOS and/or FastCGI? Please do let me know.
I cannot speak highly enough of Mediatemple and their dedicated virtual (dv) servers. I have gone through a bit of hassles as well and this was the ultimate cure.
Being locked in to a bad web host with poor support is not a good long-term solution. Rather than thinking about how much you've spent, ponder how much more you'll need to spend to get up and running. If that's a whole heap more, it might be a better call to cut-and-run.
Brendan: (mt) is what I use here and it's what I tried to push, but my employer is a bit more cautious than I am. I'm still interested in trying to get the current (new) servers working, but considering Hostway thinks of ways to rip off their customers at every corner and can't even offer good support at the same time, I'm very much agreeing with your advice.
Christian. I've been on the other side of your problems. Talking to customers that are calling up for assistance with FastCGI and PHP. The main problem is that FastCGI is mostly customized, and not a out of the box setup. Although some representatives will assist you with some things (I was one of them, cause it was easy). Anything outside of what they setup is considered administrative work, and they will charge you. One thing that I do like to mention to people in your position; with a company that has dedicated servers, Is to hire a part time/full time systems administrator. Because not only is there setup, but there is also maintenance. And by maintenance I mean updating the distribution and fixing whatever breaks afterwards, finding out why your server is attacking other servers (via cross site scripting exploits) and making sure its secure. There are many things on top as well, planning for expansion, finding problems or mis-configuration and etc.
Shoot me an email and I'll drop you my MSN and I'm sure I can help you out. If not, I can point you in the right direction.
I don't think cautious is the right word, if your employer won't perform a complete assesment of risk/return for his servers, given they are underperforming according to your estimate, while their hosting company clearly isn't helping. Good luck.
jt: I hear what you are saying but (mt) sets up all dedicated virtual servers with FastCGI by default and Plesk supports it (it's just a checkbox in the panel). It's not that unusual, IMO… but I guess the online documentation would suggest otherwise.
The thing is, I know what needs to be done to keep things secure. I can handle things on the scripting ends. I know my way around CentOS a bit. Maybe I just know enough to be dangerous. I just got my hopes up because I was able to update everything on my MediaTemple (dv) at the command line and perform all the necessary steps but I wasn't able to get very far at all with the Hostway server. I'll definitely send you an e-mail.
Jean-Francois: Good point. I guess the one thing was that they didn't start having problems until a few months ago, and so they are not quite ready to pull the plug… I sure am though, because I could tell Hostway was a ripoff very early on. I've never seen another hosting company try so hard to screw their customers… and I've dealt with quite a few. Regardless, I want to at least figure all this out, because otherwise I'll always be wondering what in the world I did wrong.
Christian Montoya: I've added you to MSN, and yes, if you do get Plesk you will get fast-cgi support. But Plesk bundles it in, and doesn't use an local/remote repositories that the distribution might have. Honestly, I had the same trouble as you and eventually after some long hours I got it to work. Tailing error.log and suexec.log will help a lot!
Ugh, updates: I got all the PHP issues for script sitting in httpdocs/ worked out but I had to open up a ticket because files in cgi-bin/ are not running with the right
gidin suexec. And on top of that, as far as I can tell this one site sitting on one server being accessed by myself only is running slower than the same site sitting on a server with two other sites while getting huge traffic… how disappointing. Right now PHP is running as CLI on the new server and CGI on the old one which makes me think that CLI is really, really slow, but my (mt) has it running as CLI and I don't see the same problem there so I'm not sure what the problem is.i might be missing something here but can't you just use PHP as an apache module rather than as CGI? much simpler and has much better performance than CGI - i think its probably even quicker then using a CGI/FCGI….
ns: My plan right now is just that, to uninstall Apache, PHP & Perl, then do a fresh install of Apache with mod_perl and mod_php, so CGI won't be used any more. I just need to verify that the user permissions can be set up properly, because I don't want to make any files on the server world-writable but so far I'm not sure how to set things up to make sure that Perl and PHP will run as group-users instead of "nobody." Any advice?
Um, to go back on that: I think I'm just uninstalling Apache, not PHP & Perl.
OK OK OK so this is what I've determined:
PHP & Perl on these servers seems to be running as mod_php and mod_perl. I'm not sure how to test if that's true but apparently PHP 5 installed as CLI is mod_php by default and I installed mod_perl with Yum which should have set it up properly.
Then, the GREAT news is that I found that Yum has a package for eaccelerator, and all I had to do was install that and it was setup automagically, so I now have eaccelerator running on this new server! Which means that the performance is good enough to go forward with migrating the site (and it took a lot less work than what it would have taken to do FastCGI or APC). So, uh, here's to getting through this mess!
if you are running mod_php or mod_perl then the php/perl scripts inherit the apache user when they are running. typically you would put the apache user in whichever group has access to the folder you wish to write too. If you are getting problems writing to folders (even if you set the permissions to 777 for example) check CentOS isn't blocking the apache user with ACL rule - i'm not that familiar with Cent but i think it does have Access control lists like that running by default….
just to make things more complicated - if you have a site with high traffic volume and you are having performance issues it might be beneficial to look at some sort of caching system, say Squid for instance as that might take some of the load off the web server
ns: Yeah I figured that out, I discovered that Plesk has its own permissions for suexec and so when you update anything related to apache or change anything you have to copy Plesk's suexec file over to the main suexec file. As of now scripts are able to write properly without having to make files 777 which is great.
I had the developers behind the site I'm currently migrating implement custom view caching so that should help a lot. I also added output zlib compression and I've definitely seen an improvement in performance. Needless to say, with any amount of optimization and each one of these high-traffic sites sitting on its own server, they should all perform much better and I won't have to worry about them anymore.
Today was definitely a much better day than yesterday. By far. And while I was figuring all these things out, I also figured out how to fix my APC setup on my personal (mt) server so now you should see all of my sites loading faster… exciting!
Load faster = good. You might also want to look at xcache but you're using APC anyways. memcached is also really good too, but you need to incorporate it into your already written code.
Suexec is nice, and if you tune it properly with fastcgi its even better. If you really want the most bang for your hardware, look at lighttpd.
On a side note, I'm working with drupal CSS templates. Just wondering if you got any documentation on tableless css, drop a link or two?
jt: Apache is faster that lighttpd
I'm sorry I can't help you with Drupal, I've never used it. I heard some people are working on Blueprint CSS themes in Drupal, but I don't know if anything has been released yet.
To revive an old topic. You should consider using apaches mpm_worker with FASTCGI, almost comparable to lighttpd speeds.
Looks like hatred for hostway is pretty common on the web. I'm in the process of moving my sites away from them, and they are not making it easy on me.
I've already prepared a nice mean rant on my personal blog, that I will post just as soon as I've cancelled my Hostway account.
MindExpansion.com sucks a big one! Their internet connection is always down!