Useful Little Things

This is a reprint of an old article I wrote for OISV.  I have since switched from GoDaddy hosting to WebFaction, for reasons of performance.

Lesson 1: Know the Environment

Before I signed on with GoDaddy, I did a bit of research on their Linux shared host offering. Their “Deluxe Plan” provided just what I needed: Linux, Apache, Tomcat, MySQL, and PHP. Not only did I know and understand each of these technologies, but I already had them installed at home from prior projects.

Unfortunately, I forgot to check one little detail: the version numbers. You see, GoDaddy is a little behind the technology curve. Not far, but just far enough to have caused me trouble. While I was running MySQL 5.0 at home, GoDaddy was running MySQL 4.0. Among other things, MySQL 5.0 (actually 4.1.1) introduced a really cool feature called Query Expansion, which is great for searching things, like forum postings.

It only took a few minutes to rewrite my queries to remove Query Expansion and become compatible with MySQL 4.0. But there were several others problems I ran into because of version number differences (TRUE/FALSE keywords, BOOLEAN data types, etc.). And every minute I spent fixing these little problems was a minute I wasn’t adding features to my site.

In short, I was reminded that the devil is in the details. And software has lots and lots of details. Starting with version numbers.

Lesson 2: Know the Vendor

I chose GoDaddy for a couple of reasons, the first being price. The second reason I chose GoDaddy was because they had been around for a while and I wasn’t worried about them disappearing without warning. The third reason was their support documents. GoDaddy has quite a bit of support material on-line for its customers. In retrospect, it’s not as much as it seemed because some of the information is duplicated or out-of-date.

Early on, I also discovered that phone support, while not exactly stellar, is decent. You usually won’t be kept on hold for too long. Also, the CSRs won’t transfer you when you ask a question they can’t answer. They’ll try to clearly articulate your question back to you, then they’ll go ask second level support for you. This means you won’t have to keep repeating your problem to each new person. The catch is that first-level support has no real idea what Tomcat is, much less Java. Any question that delves into those topics will immediate send the CSR off to ask their “Tomat/Java Guy.” Thankfully, I’ve gotten mostly satisfactory answers from this process.

That said, let me tell you what I think of their email support. It’s worthless.  I will never use it again. Was I unclear?

Recently, I noticed my web application was down (returning 500 error codes). I checked my static content and saw it was working fine. Conclusion: Apache is running, Tomcat is not. By the time I got around to contacting GoDaddy — about two hours later — the site was back up. I sent a short email to technical support clearly stating that Tomcat had been down (and cited all the checks I had made) and that I needed to know what happened. Is this something I could expect to occur regularly? Was it scheduled or unscheduled maintenance?

I received a response within the hour. The email from GoDaddy suggested that I should be using the GoDaddy nameservers and directed me to the appropriate support pages. I read the email three times and still couldn’t see how the CSR could possibly have thought that his response had anything to do with my request. I figure one of two things happened. My request hit on some keywords that auto-generated the response, or the CSR that processed it really didn’t care one way or the other if they answered my question and just grabbed a random response. Either way, I don’t use email support anymore.

So, the point is that you should quickly learn the limits and behaviors of your vendor. In my case, GoDaddy == telephone support, GoDaddy != email support. Learning and adjusting to your provider’s limits will save you frustration later on.

Lesson 3: Start Small

I’m very cost-conscious and I saw no reason to spend money on the hosting plan until I actually had something tangible (like a website) to put out on it. So, I spent the first three months building quite an impressive base application, including user registration, forums, FAQs, and various other dynamic, database-driven content.

You can probably guess what happened when I deployed for the first time. Nothing. Nada. Zip. I couldn’t get a single web page to come up outside of the static content.

I spent the next four weeks untangling a giant mess of problems and incompatibilities. To begin with, I was using a standard context.xml file in my WAR file to give Tomcat additional deployment instructions. Not supported on shared hosting. I was also using Log4J to write messages to a /logs directory. Java cannot write to any folder other then /tmp, which is public and shared. I was using the same name for the WAR file every time.

You must delete your exploded WAR file directories before re-deploying, which you cannot do if anyone has accessed the web application since the last Tomcat restart which happens (most) every night.

I finally ended up creating a Hello World web application in which I incrementally added feature after feature to “feel out” the system and determine what actually worked and what didn’t. My context.xml file went out the window and Log4J got redirected to write to the database. I haven’t found an elegant solution, yet, that resolves the WAR file naming issue and still allows me to maintain some level of SEO.

The lesson here is simple. Sign up for the hosting plan right from the beginning. It just makes sense to spend an extra thirty or forty dollars up front to save time on the back-end. Next, start with a small, basic web application about the size of a Hello World app. Incrementally add each your features, with a deploy-and-test between each one. By features, I mean things like logging, third-party tools, and architecture/framework changes.

Lesson 4: Start Early

One of the most frustrating aspects of the shared hosting account is the time lag for things to take effect. The fact that web applications can only be deployed nightly when Tomcat restarts is a prime example. The Hosting Manager Control Panel is another. Any changes you are allowed to make must be done through the Control Panel. This includes setting your 404 page, managing your FTP users, activating Apache error logs, and managing your MySQL database(s). Each change you make will be greeted with a “change pending” response. For some reason, none of the changes you make occur real-time. Some may take a few minutes, other a few hours. I’m partially convinced that a CSR is hiding behind the Control Panel manually implementing each of your requests.

This behavior is reflected throughout GoDaddy’s systems (sharing hosting, email, domains, etc.). Therefore, as soon as you have an account with GoDaddy, there are several things I recommend setting up in advance. This will give everything a chance to become effective before you actually need it.

First, get the shared hosting plan active and make sure Java support is enabled. Next, set up the following using the Control Panel:

• point your domains to the shared host,
• create your FTP user(s),
• create your MySQL database(s),
• set your 404 error behavior,
• and create your sub-domains (i.e. -
• blog.domain.com, support.domain.com, etc.).
• Next, set up your free and paid email accounts, as well as, your email forwards.
• Lastly, set up a simple mod_rewrite script on your host.

You may find at some point that you will need to use mod_rewrite and some people have reported that their servers did not have it initially enabled.

Setting all this up at the start will require a bit of planning, but at least you won’t have to wait a indeterminate amount of time for the “pending change” to take effect.

Lesson 5: Keep It Simple

I’m used to working in environments where you have complete control over the systems involved, or you at least have someone you can count on that has complete control over the environment. So I found it a bit disheartening to hear things like “you can’t do that” or “that’s not allowed” from the GoDaddy support team. Writing to the file system from Java is one glaring example. Manual deployment to Tomcat is another.

The only way to deploy a web application under Tomcat is to copy the WAR file to the file system and wait for Tomcat to be restarted, which happens most every night.
I say “most every night” because I’ve experienced a few incidents where Tomcat did not restart (or the web application did not deploy… the support team was unable to say which it was). The only way to un-deploy a web application is to delete the exploded directory structure, which I mentioned earlier cannot be done if some has accessed even one of the pages in the application.

I say all this to emphasize how difficult and time-consuming it can be to put your Java code out on the server. Therefore, you should make every effort to remove everything possible from the WAR file (i.e.- the web application). Static content should be copied directly onto the server, not packaged in the WAR file. Dynamic content should be placed in a database to be read and displayed by the web application, so when you need to change some wording on the web page, you won’t have to redeploy. In fact, you may want to consider moving all the content into a database and allow the JSPs to pull the text using custom tags.

Disclaimer: I have not checked the performance of the GoDaddy web/database servers to know if this will perform well under heavy loads.

In short, your web application should contain the bare minimum required to operate. Other content, such as style sheets, static pages, dynamic text, etc., should be removed and placed in an appropriate location. This will minimuze the number of deployments you’ll need to perform and will give you more immediate control over changes to the site.

top

Lesson 6: Keep Good Records

My first week with GoDaddy was spent mostly staring at blank web pages, wondering why they didn’t load, and looking for log files, wondering why they were never created. Of course, it turned out that my logging infrastructure was the culprit in most of my pain. Once I disabled it, I started actually seeing some real errors and was able to perform some real troubleshooting.

Not all errors, though, show up in the rendered web page. That’s why you need logging. Despite my earlier trials, I still fully support using Log4J. In this case, however, you’ll need a database appender such as the one that ships with Log4J or the one maintained by Danko Mannhaupt. This will allow you to view the log entries even if the web application refuses to come up.

Of course, if you cannot acquire a database connection from the very start, a database appender will not do you much good. Therefore, I recommend implementing a logging framework with a failover to the file system in the event the database cannot be reached.

As I mentioned earlier, the only location Java can write to is the public, shared /tmp folder. So, it is a simple matter of creating and appending to your own log file in that location. Being the paranoid type that I am, I recommend using a symmetric encryption algorithm to keep prying eyes from reading your files. You should also delete your log files on a regular basis, rather than relying of GoDaddy to do it for you.

In the end, all this logging will do you no good if you haven’t established a proper logging policy for yourself. I know, I know. Having a logging policy sounds so… corporate. What I really mean is that you should decide where you’re going to do logging (database and/or file), when you’re going to write to the log (info vs. debug vs. fatal), and what you’re going to log (I really like including usernames and session information). Even further, you should use logging statements consistently throughout your entire application, because the one class you neglect to include logging in will be the one that blows up next.

So, in summary, I highly recommend putting up front effort into a logging process that both supports your debugging needs and fits within the limitations of a GoDaddy shared hosting site.

Lesson 7: Work Around the Limits

I was quite disappointed to discover that you cannot connect directly (or indirectly) to a GoDaddy MySQL database. The best tool offered by GoDaddy is phpMyAdmin 2.7. This is a very respectable tool and I tip my hat to the development team. That said, however, it’s a terrible way to administer a production database.

That’s why I built a couple of features into my web application. First, my web application will create or upgrade the existing database when the application starts. This eliminates the need for separately deployed database updates. It also means I don’t have to use the phpMyAdmin interface very often.

The second feature I built into the application is a data maintenance page. I can add, edit, or delete data for certain parts of the application, such as the user and FAQ lists. Other features of the web application are “in place” editing, such as in the user forums. I can moderate forum postings while reading them, as long as I am logged in as an administrator.

The third feature I included is a page to load, decrypt, and display the log file in the /tmp folder. Of course, I have a stand-alone application to do the same in case the web application is completely down. I simply have to grab the file via FTP and run it locally.

The common thread running through each of these features is they overcome some limitation of the GoDaddy setup. Therefore, I recommend to you that you build a suite of tools customized to your needs, which help you overcome the obstacles inherent to your environment.

Lesson 8: Plan for Growth

Other developers have asked me why I didn’t write my application in PHP or .Net. After all, I know each language and there are plenty of free tools I could use for both. My short answer is that I am comfortable with Java as a highly-scalable, enterprise-level platform. PHP does not naturally encourage good programming and object-oriented design practices. As for .Net, it only runs on Windows servers under IIS and I prefer to use the Linux/Apache combination which is “free” and reasonably reliable.

The choice of Java also allowed me to design a web application that operates under Tomcat, yet has an upgrade path to J2EE. I modeled the business layer to mimic some of the architecture found in a J2EE application, such as session beans. Later on, I can easily migrate the web application to full enterprise application status with minimal changes. This will be especially important when the activity level of the site outgrows shared or virtual hosting.

Having an upgrade path for the website is just one aspect of planning for growth. The fundamental idea here is that you should expect your website to grow in terms of visitors and functionality. Keep this in mind as you design each component and/or feature remembering to allow for growth.

Lesson 9: Monitor

I discovered early on that GoDaddy Web Statistics are mildly interesting, but completely useless for actually monitoring a website site. That being the case, I chose to sign up for Google Analytics. It was quite simple to add the Google Analytics JavaScript to the bottom of each page in the web application, especially since I was using Struts/Tiles to construct the JSP pages. Google Analytics allows me to track visitors and navigation with great detail.

On the other hand, I also needed a way to track the status of the site, to make sure everything was working properly. I elected to schedule a simple JMeter test to run every so often from another server. I can review the test results to see if any downtime occurred.

As a side note, you should also create and maintain a sitemap for your website. Google has excellent resources on the topic. A sitemap will help Google understand and crawl your site. You can also use it to mark off-limits areas.

In short, it is your responsibility to plan for and track activities on your site. The only thing worse than a site with no visitors is a site that you can’t tell if it has any visitors at all.

Lesson 10: Automate

I’m basically lazy. So, I automate anything and everything. Automation saves me time and effort and reduces the chance that I will screw something up. Here’s a few automation tips I’d like to offer.

Ant Builds.
I use Ant scripts to build my WAR files. In fact, my Ant script builds separate WAR files for my home server and for the GoDaddy server. My Ant scripts stop, deploy, and restart Tomcat. They clear old logs files. In short, they do anything I’ve had to do more than once.

Script-fu Buttons.
I don’t pay for buttons for my website and I don’t steal graphics from others. I use a simple little script for Gimp called Simple Beveled Button. In fact, I keep a copy of the complete script required to create the buttons. In the event I change color schemes, I can re-run the script with the change and presto! New buttons.

JUnit.
I don’t write software that isn’t backed by a unit test. Period. Unit tests are simply a necessity. And JUnit is a simple, complete, and effective way to build them.

Lesson 11: Get the Right Tools

I use a fairly standard set of tools for Java development: Eclipse, Subversion, FireFox, FileZilla, Ant, JUnit, JMeter, and GIMP. Eclipse is an excellent Java IDE with tons of plug-ins available. Subversion is, in my opinion, the best version control system available. I’ve used CVS, SourceSafe, and PVCS, and Subversion handily outperforms then all. I use FireFox with the Web Developer plug-in, which is invaluable for troubleshooting problematic HTML. I use FileZilla for FTP operations and it handles every protocol I need. I use Ant for all my build and JUnit/JMeter for all my testing. Lastly, I use GIMP for all my image editing.

You can see that all of these tools are either free or open source, making my toolbox extremely cost effective. However, I still have a few tools I’ve paid for, simply because the right tool will pay for itself many times over. For example, I use UltraEdit for all text editing outside the IDE.

Lastly, I understand my limitations as a developer, my biggest being a lack of artistic skills. Specifically, I couldn’t design a clean, beautiful website to save my life. Therefore, I’ve pulled various templates from the OSWD site. I’ve learned to avoid the “most popular” templates after seeing another site nearly identical to mine in look-and-feel, because we had used the exact same template.

Ultimately, I recommend building a toolbox that you are comfortable with, using a blend of free and open source software, commercial software (where warranted), and knowledge sites (like OSWD.org).

Lesson 12: Search Engine Optimization (SEO)

I wanted to make sure that Google, Yahoo, and MSN Search could easily crawl and catalog my site. So, I used all the standard SEO technique: proper titles, H2 tags, etc. However, I ran into a little problem with the pages in the web application itself.

I’ve already pointed out how you cannot deploy over an existing web application. Initially, I renamed the web application’s WAR file on each deploy, resulting in the web application begin found under http://www.somedomain.com/app1, then http://www.somedomain.com/app2, then http://www.somedomain.com/app3, etc.

I realized, though, that the search engines would not appreciate this kind of moving content. So, I set up a mod_rewrite script to redirect http://www.somedomain.com/app requests to http://www.somedomain.com/app9 (or whatever it was this week). The problem is, Apache handles the URL rewrite, while Tomcat handles the web application requests. So, rewritten URLs never made it to Tomcat, resulting in more blank pages.

My final solution is not terribly elegant but it is effective (if you have a better idea, send me an email via the website below). I decided to deploy the web application under a standard URL such as http://www.somedomain.com/app with a permanent redirect from the server root (http://www.somedomain.com/index.html). Any visitors, including search engines, will bounce from http://www.somedomain.com/index.html to http://www.somedomain.com/app. Then, the new version would be deployed under http://www.somedomain.com/app-temp with a temporary redirect from the server root (http://www.somedomain.com/index.html). A day later I would be able to delete the web application from http://www.somedomain.com/app and deploy the new version. Lastly, I would put back the permanent redirect from the server root to http://www.somedomain.com/app.

If you actually kept up with that, then I’m impressed. If not, then look at it this way… New versions of the web application are temporarily installed in a new folder where I temporarily direct all traffic until I can replace the web application in the “standard” location.

In short, if your site depends on traffic from search engines, you will have go beyond simple SEO. You’ll need to put extra thought into how to construct a search-engine friendly site for the long term.

Lesson 13: Blog It

There are dozens of experts out there telling you to blog, blog, blog. I happen to agree with them. I’ve noticed that most of the traffic my site is receiving is coming from one or another of my blogs.

As for what to blog about… that’s up to you. Just keep in mind that the information in the blog should ultimately be targeted at the intended audience for your site. Give them good information on topics you know something about.

For example, I know this guy who has some fairly specific experience building web applications for deployment on GoDaddy…

A GoJava Roadmap

I wanted to summarize as much of the information in this article as possible. The result is my GoJava Roadmap available in PDF or PNG format.. In the roadmap, you’ll find step-by-step guidelines on building your next Java web application for GoDaddy.