Apache mod proxy + Jboss + virtual hosting

I've just spent countless hours migrating from Orion Application server, java 1.4.2 and MySql 3.2 to latest version of apache, jboss 5.1 and Postgres.

Now, migrating the database was pretty easy. 1 gig of data, ~10 million rows total, performance difference to foreign-key-less MySql 3.2 vs. Postgres 8.4 is really _a lot_.

Now, what I wanted to share is what I personally spent two days configuring, so that someone else might be spared the trouble. My setup is as follows:

Several domains, for the purpose of this example I shall call them www.firstdomain.com and www.anotherdomain.com. Both are Java-based sites running on jboss, basic Java backend (No fancy EJB stuff yet, coming to that later ;)), and JSP. Both run on same IP. Custom URL rewriting involved. apache configured as front-end only, to forward requests to the sites, nothing more. The sites are run on jboss, bound to localhost:8080 ( so you don't have to tweak jmx-console, web-console or the such at all ! ). Contexts localhost:8080/firstdomain and localhost:8080/anotherdomain

Now, the setup. With working sessions too (ie, the sessions won't break in between requests. Called Sticky sessions).

First, set up you jboss and the sites. Make sure you can access them locally. Nothing more needed to tune on jboss.

Next, install apache. My version in this example is 2.2. In httpd.conf, enable mod_proxy by uncommenting the necessary lines, ie. mod_proxy.so, mod_proxy_http.so.

As the sites run on the same IP, we need to set up name-based virtual hosts. Add the following line to httpd.conf:

NameVirtualHost *:80

You can replace * with the IP if it makes you feel better. Now, the example virtual hosts:

<VirtualHost *:80> 
ServerAdmin [e-mail] 
ServerName www.firstdomain.com 
ServerAlias firstdomain.com 
proxyPass / [localhost:8080...] 
proxyPassReverse / [localhost:8080...] 
proxyPreserveHost On 
proxyPassReverseCookiePath / / 
ErrorLog logs/firstdomain-error_log 
CustomLog logs/firstdomain-access_log combined 
</VirtualHost>

<VirtualHost *:80> 
ServerAdmin [e-mail] 
ServerName www.anotherdomain.com 
ServerAlias anotherdomain.com 
proxyPass / [localhost:8080...] 
proxyPassReverse / [localhost:8080...] 
proxyPreserveHost On 
proxyPassReverseCookiePath / / 
ErrorLog logs/anotherdomain-error_log 
CustomLog logs/anotherdomain-access_log combined 
</VirtualHost>

Now, lets break this into bits and explain what's what. First, ServerName and ServerAlias are the domains that the virtual host will respond to.

proxyPass will forward the request to these domains to the address localhost:8080/firstdomain/ and so on. Ie. to jboss.

proxyPassReverse will retain the domain's name in the request headers, even though the request goes elsewhere. So your visitor doesn't actually know he's being forwarded to jboss.

proxypassPreserveHost will retain the request domain in the headers, even though the request goes to another domain, localhost (this is important cookie-wise. If it's not set, cookies set on your domain are not visible, because the headers will return a different domain, localhost).

proxyPassReverseCookiePath will retain the original path in cookies, even though the contexts have /firstdomain and /anotherdomain. Very important for sessions, and sessions will not stick unless this is specified.

I hope this will help people in the future, so I posted it here, because Google and others tend to find stuff on Webmasterworld ;) 

 

jdMorgan




msg:4012419 8:54 pm on Oct 23, 2009 (gmt 0)
Thanks for posting!

Jim 

tk_meyer




msg:4013281 4:09 am on Oct 26, 2009 (gmt 0)
Thanks for posting this, it is very helpful. I am trying to configure a system similar to yours with a slight variation.

I will need to parse the URL to determine which VirtualHost to forward the request to. The url will look something like [webservice.blaa.com...]

I will need to forward based on the last 4 digits in the URL, however, I will not know which jboss instance to forward to until I do a database lookup to see which server this resource is located on. I am serving up web-services that run against several oracle back-ends which are segmented into virtual private databases, the resources can and do get moved.

I have not found any documentation on the apache site that suggest what might be the best way to go about this. I have looked at dynamically writing out an .htaccess file, but have read where this might have performance implications.

I have also looked at the apache Camel project, but I do not think that this warrants a message bus.

Has anybody configured a system similar to this? Any suggestions and/or advice would be greatly appreciated. 

jdMorgan




msg:4013594 4:06 pm on Oct 26, 2009 (gmt 0)
Take a look at mod_rewrite's RewriteMap. You can use that to call a script during the URL-to-filename translation phase of the apache API, and then either rewrite to the result of that lookup (if the path is on this server), or do a reverse-proxy through-put to the correct server using the [P] and [PT] flags.

Generally, the script called by using a RewriteMap will be written in PERL, but it could be anything that can support reading your database. Be very careful, though; This script must never be allowed to 'die' because only one instance of this script is ever started or executed. If it dies, your server(s) won't work.

RewriteMaps must be defined at the server config level, but can then be accessed by mod_rewrite code in both server config files and in .htaccess files.

Jim 

tk_meyer




msg:4013599 4:13 pm on Oct 26, 2009 (gmt 0)
Thanks Jim for the speedy reply. I will give that a shot and post how it goes.

TK 

Vegeta




msg:4020472 4:50 pm on Nov 6, 2009 (gmt 0)
Thanks for posting this thez, it is very helpful to me as I am setting up a similar machine.

My big question now is if set up apache to run SSL, when the connection is redirected to "firstdomain" or "anotherdomain" will the ssl conection carry over or will the connections to firstdomain and anotherdomain be unsecured?

TYIA 

MichaelBushe




msg:4041283 11:40 am on Dec 11, 2009 (gmt 0)
Vegeta - on your question, I believe the apache docs say you can't use name based virtual hosting with SSL. I think because the domain is encrypted. MichaelBushe




msg:4041291 12:28 pm on Dec 11, 2009 (gmt 0)
I've also been bashing on this for a while. Thank you for your post, but I still can't get this work.

My two domains are bushe.com and eventbus.org. When I got to [bushe.com,...] my browser gets a 302 and gets redirected to 
[bushe.com...]

That results in a 404 with the description: The requested resource (/bushebushe/) is not available.

Are you getting the 302 redirect? Since it's a proxy (and to localhost), that's not what I'm expecting. And you said the browser is not supposed to know.

My apache configuration is almost exactly like yours, maybe there's something wrong in jboss?

The bushe.war dir has a <context-root>bushe</context-root> in jboss-web.xml, similar for eventbus.war.

Here's my httpd.conf:

NameVirtualHost *:80

<VirtualHost *:80> 
ServerName www.bushe.com 
ServerAlias bushe.com 
proxyPass / [localhost:8080...] 
proxyPassReverse / [localhost:8080...] 
proxyPreserveHost On 
proxyPassReverseCookiePath / / 
ErrorLog logs/bushe.com-error_log 
TransferLog logs/bushe.com-access_log

# DocumentRoot /var/www/bushe.com 
# Options Indexes +FollowSymLinks 
</VirtualHost>

<VirtualHost *:80> 
ServerName www.eventbus.org 
ServerAlias eventbus.org 
proxyPass / [localhost:8080...] 
proxyPassReverse / [localhost:8080...] 
proxyPreserveHost On 
proxyPassReverseCookiePath / / 
ErrorLog logs/eventbus.org-error_log 
TransferLog logs/eventbus.org-access_log

# DocumentRoot /var/www/bushe.com 
# Options Indexes +FollowSymLinks 
</VirtualHost>

The bushe.com domain works if I use [localhost:8080...] instead of [localhost:8080...]

TIA 

MichaelBushe




msg:4041309 1:23 pm on Dec 11, 2009 (gmt 0)
It seems jboss is sending the 302, I'm not sure why. Since this is not apache, I'll take it outside this forum: 
wget [localhost:8080...] 
--07:55:09-- [localhost:8080...] 
Resolving localhost... 127.0.0.1 
Connecting to localhost¦127.0.0.1¦:8080... connected. 
HTTP request sent, awaiting response... 302 Moved Temporarily 
Location: [localhost:8080...] [following] 
--07:55:09-- [localhost:8080...] 
Connecting to localhost¦127.0.0.1¦:8080... connected. 
HTTP request sent, awaiting response... 200 OK 
Length: 2733 (2.7K) [text/html] 
Saving to: `index.html' MichaelBushe




msg:4041311 1:29 pm on Dec 11, 2009 (gmt 0)
DOH! I figured it out. I did not have this trailing "/" on the proxyPass: 
WRONG:proxyPass / [localhost:8080...] 
RIGHT:proxyPass / [localhost:8080...]

Having been bitten by the trailing slash so many times with mod_rewrite, I should have caught that earlier.

THANKS FOR THE POST! 

tk_meyer




msg:4060814 5:52 pm on Jan 14, 2010 (gmt 0)
Hi Jim,

I have having a tough time. I have read and reread the apache doc for mod rewrite and cannot seem to get my rewrite to work for post requests, the get requests work fine. Would you mind looking at my rules to see if anything jumps out at you?

ServerName vmQADev05 
Options +FollowSymlinks 
RewriteEngine on

RewriteMap club prg:/var/www/php/clubs.php

RewriteLock /etc/httpd/logs/rewrite.lock 
RewriteLog /etc/httpd/logs/rewrite_log

RewriteCond %{REQUEST_URI} ^/ws/(.*)/([0-9]{4})$ [NC] 
RewriteRule ^/ws/(.*)/([0-9]{4})$ [10.xx.xx.xx:${club:$2}...] [R,L]

I can proxyPass and both the POST and GET work. My problem is I do not which jboss to forward to until I do a db lookup for matcher group $2. Does anybody have any suggestions or point me to a code sample? 

jdMorgan




msg:4060933 8:26 pm on Jan 14, 2010 (gmt 0)
The RewriteCond is utterly redundant. Since the RewriteRule is looking at exactly the same thing, get rid of that RewriteCond.

See the [P] flag for RewriteRule, since you don't want a redirect here, but rather a reverse-proxy through-put.

Jim 

tk_meyer




msg:4060977 9:16 pm on Jan 14, 2010 (gmt 0)
Thank you for the help!

I ended up with this and it works like a charm:

RewriteRule ^/ws/(.*)/([0-9]{4})$ [10.xx.xx.xx${club:$2}...] [P]

I had to remove the colon from the path and return it from my script, so instead of returning 8080, the script returns :8080 � what ever port myjboss is listening on. Otherwise I was getting a DNS lookup error, because the return value from the script was not getting substituted in the path.

Go figure, I think I have fried many brain cells :).

tk 

jdMorgan




msg:4061077 11:54 pm on Jan 14, 2010 (gmt 0)
That's kinda weird. I suspect the colon may have cause incorrect parsing of the "${club" map invocation.

Did you try escaping it? Escaping is rarely required in substitution strings, but you may have just found one of those rare requirements... :)

Just wondering if you tried: 

RewriteRule ^/ws/(.*)/([0-9]{4})$ http://10.0.0.1[b]0\:${c[/b]lub:$2}/abc-web-services-1.3.0/$1/$2 [P]

Jim 

tk_meyer




msg:4061092 12:26 am on Jan 15, 2010 (gmt 0)
I did try to escape the colon, but that did not work either. I think that is what made it difficult to figure out what was going on.

Thanks for all your help :)

tk