Tomcat Vs GlassFish (Which is faster for APEX and ORDS)

Introduction

Running ORDS in standalone mode is not supported for production environments. This means you need to run ORDS in a J2EE Web Server. At time of writing, the three supported J2EE web servers are Weblogic, Tomcat and GlassFish. Of these, only Tomcat and GlassFish are Open Source (read free). Oracle's view on support is that it supports running ORDS on these web servers but does not support the web servers themselves. This means that if you have issues with ORDS on these servers you will have to prove ORDS is causing the issue and not the web server.

---

Having said that, both Tomcat and GlassFish are capable web servers and I have a number of customers that have been running ORDS on them in production for many years, without any issues.
When choosing between these web servers, I typically recommend GlassFish unless the client has a preference for Tomcat. This is based on my personal preference and ease of use as opposed empirical evidence. I decided it was time to run a performance test to see if one performed significantly better than the other.

---

Tests Performed

The goal of the speed comparison is to compare the two web servers under moderate loads. In view of this the tests are not overly complex or load intensive. The load will be generated from number of simultaneous 'users' and repetitions. The idea being to compare them not break them.

​APEX Page Render
A basic APEX public page with three regions. A region with a classic report, a region with 5 form fields and a static text region. This test is aimed squarely at how the web server handles APEX page requests. The test is run 25 concurrent threads and repeated 10 times for a total of 250 transactions. The siege (see below for details on siege)  command run was:
siege -c 25 -r 10 -d 0.1 -m "ApexDynamicPage" -f /home/ec2-user/siege/ApexDynamicPageURL.txt

APEX Page Used in Tests

ORDS REST Get Service
A basic GET service with the following SELECT (no OAUTH security enabled):

select owner,object_name
from   all_objects
where  object_type = 'SEQUENCE'
and    owner  = 'MDSYS'

This is the same SQL used in the APEX page. Under ORDS 3.X calling a REST service does not rely on APEX at all. This test is focused purely on throughput from a REST perspective. The test is run using 25 concurrent threads and repeated 10 times for a total of 250 transactions. The siege command run was:
siege -c 25 -r 10 -d 0.1 -m "ORDSRESTGet" -f /home/ec2-user/siege/ORDSRESTGet.txt

Minified CSS File
I used the minified version of the apex ui css 'apex_ui.min.css'. The file is located in the Apex images folder and referenced in the ORDS i.war file as: i/css/apex_ui.min.css. The file is 108,694 bytes. The test runs in 100 concurrent threads and is repeated 5 times for a total of 500 transactions. The siege command run was:
siege -c 100 -r 5 -d 0 -m "ApexStaticCSS" -f /home/ec2-user/siege/ApexStaticCSSURL.txt

​Static HTML File
I created a small static HTML file and placed it in the root of the respective web server. The file is 3,711 bytes. The goal of this file is raw throughput of the web server as APEX and ORDS are not involved in serving this page at all. The test is run 100 concurrent threads and repeated 5 times for a total of 500 transactions. The siege command run was:
siege -c 100 -r 5 -d 0 -m "StaticHTML" -f /home/ec2-user/siege/StaticHTMLURL.txt

Test Conditions

Servers
All of the servers used were running on the us-west availability zone on AWS. They were all running the AWS flavor of Linux except the DB Server which was running OEL.

• DB Server - t2.large (2 cpu, 8gb Ram)
• App Server - t2.small (1 cpu, 2gb Ram)
• Test Server - t2.small (1 cpu, 2gb Ram)

​Software
I used the latest stable versions (at time of writing) of all the software involved (except the DB):

• Database 12c (12.1.0.2.0)
• ORDS 3.0.8
• APEX 5.0.4
• JDK8 (1.8.0_111)
• GlassFish Server (Web Profile) V 4.1.1
• Tomcat 8.56

Key Configurations

ORDS

• jdbc.InitialLimit = 25
• jdbc.MinLimit = 25
• jdbc.MaxLimit = 25

GlassFish

• Set logging level to WARNING
• JVM Xms and Xmx = 1024MB
• JVM -server option
• Caching Off
• Compression Off
• Thread Pool Size = 100
• max-connections = 100

Tomcat

• Set logging level to WARNING
• JVM Xms and Xmx = 1024MB
• JVM -server option
• Caching Off
• Compression Off
• Thread Pool Size = 100

Load Test Software

In order to eliminate network as much as possible, I did not want to install the load test software on my laptop. This ruled out desktop tools such as JMeter. In view of this I installed Siege on an AWS EC2 server in the same region as my App Server and DB Server. For a command line tool, Siege comes with a number of features included with a number of the more sophisticated tools. I used version 3.1.4 of Siege in my tests.

Test Method

• Start GlassFish
• Run each of the four tests (throw away results, server warmup)
• Wait 1 Minute
• Run the four tests twice, average the two results (document results)
• Stop GlassFish
• Start Tomcat
• Run each of the four tests (throw away results, server warmup)
• Wait 1 Minute
• Run the four tests twice, average the two results (document results)

The Results

Thanks for sticking with me this far! I do believe, however, it is important to lay out the method I used so you can understand what was and what was not tested. This allows you to more easily apply the results to your situation. First of all, let's look at the raw results (winning metrics are highlighted in green):

Summary

As you can see, generally it was a pretty close race. This is not totally unexpected. For ORDS, these web servers are acting as a J2EE container and they are both using the exact same version of the Java JDK.

Apex Page
Generally Tomcat seems to be about 3% faster than GlassFish except for in the area of concurrency where the difference was only 1.45%.

REST Service
Again Tomcat is the victor, this time by a larger margin of around 7% for response time, elapsed time and transaction rate. The two are almost exactly matched for throughput (mb) and concurrency.

Compressed CSS
GlassFish fights back here with a significant win over Tomcat of between 12% and 15%. I believe this shows that GlassFish is much better at serving larger files (the css file is about 100k).

HTML File
Once again Tomcat wins by around 3% except for the area of concurrency where they are again almost exactly matched.

Overall
One deduction from the above (and I am sure you have your own) is that the two are very well matched except that Tomcat seems to be better at thread handling. What I mean by this is that it is getting through the volume of transactions and switching between them faster. But, when there is more work to do (in the case of the larger css file) GlassFish is performing better.

Conclusion

So which web server should you choose? Based on the results of my testing, while Tomcat seems to be generally slightly faster, there is really not much in it.

While not having a clear winner seems like a bit of a let down (it was for me), all is not lost. It does mean that I can now provide my 'client preference' advice with some empirical evidence to back it up.