JMJ CLOUD
  • Home
  • Projects
  • Blog
  • About Us
  • Contact Us

Our Blog

RIP Exadata Express, where do I run my RAD stack now?

9/12/2019

0 Comments

 

Introduction

APEX Hosting on AWS RDS
​As you may be aware, Exadata Express is coming to the end of its life. If you like us have found this platform both useful and affordable, then you will be wanting to know what’s next? Where in the cloud can I deploy my RAD (REST, APEX, Database) stack now? In this post I will explore an alternative which we have implemented here at JMJ Cloud.

Background

​We have been using Exadata Express and JMJ Cloud for more than 2 years. During that time, we helped five of our clients implement Exadata Express. Use cases ranged from custom cloud point solutions to ERP Cloud PaaS extensions. I have also blogged about this platform on a number of occasions. Exadata Express offered a highly functional PaaS solution for as little as $175 a month. We (and our customers) were very happy with it. In fact, I thought this product was a sign that Oracle had finally turned the corner to becoming a major cloud player. It provided a very low barrier to entry with more feature rich alternatives ready and waiting when your business grew into them. This is essentially how AWS became so successful. One day you start with half a dozen EC2 servers for a couple of hundred dollars a month and a few years later you are spending a million dollars a month (think Netflix). Oracle on the other hand typically starts by charging a lot and then work their way up from there.

How did we Get Here?

A month or so ago, our Salesperson emailed us… Here is where I need to pause for a small rant. Why on earth, if you are selling automated cloud solutions should I ever have to deal with a Salesperson? I want a server; I know what size server I want, and I know how much I want to pay for it. From my perspective, having a Salesperson in the middle of this is expensive from Oracle’s point of view and extremely frustrating from my point of view. OK, back to the story… Our Salesperson reached out and said we could not renew our Exadata Express contract and oh, by the way, you have a month to get off the platform. After a little discussion (i.e. an hour of my time on the phone) we converted that 1 month into 3 months. We bought ourselves some runway, but we still needed another platform. Enter the Salesperson (again). After another one hour of my time he proceeded to explain the wonders of the Autonomous Database Cloud Service. Don’t get me wrong, this service shows a lot of promise and it is going to be great for our larger customers. At more than $1,200 per month, however, it is not going to fly for us or our smaller customers wanting to run the RAD stack in the cloud. It is nearly 7 times more expensive than Exadata Express. If you wanted to use Autonomous Database with APEX for a Cloud ERP extension and you need a DEV, TEST and PROD instance that is $3,600+ per month instead of $600 / month.
 
To add insult to injury, there is not even a button push migration path from Exadata Express to Autonomous Cloud. This means that even if you have the money to pay for Autonomous Cloud, you are going to have to fork more money to migrate.

The Alternative

APEX Hosting the Alternative
I would love nothing more than to say I whole heartedly recommend folks move to the Autonomous Database Cloud Service but that is not where I am going. The answer to our problem was in a service we have dabbled with before which is Amazon’s RDS (relational database service) service along with an EC2 (elastic compute) server running ORDS. I have never really been completely happy recommending this approach for the RAD (REST, APEX, Database) stack before, two things have changed to make this a much more appealing approach.
  • You can now perform a full ORDS install on an RDS instance. This is thanks to new functionality in ORDS 19.1 which allows you to run the install as a user other than SYS. This is especially important now that REST services are no longer available in APEX alone.
  • AWS have started making the latest version of ORDS available on RDS much sooner than they used to. For example, as of right now, September 2019, you can run ORDS 19.2 and APEX 19.1 on RDS, whereas Exadata Express is running ORDS 3 (ouch) and APEX 18.2.

Architecture

​The diagram below shows the architecture we are using at JMJ cloud.
Picture

Cost Breakdown

Here is a cost breakdown for the above architecture.
APEX AWS Cost Breakdown
Notes
  • We did not go for provisioned IOPS (input output per second, A measure of disk speed) which means we get burst-able IOPS. This is fine for short bursts of activity but if you expect prolonged volume activity, you will want to pay for provisioned IOPS which will increase the cost.
  • RDS service pretty much doubles as you go from db.t3.small (2GB RAM, 2 VCPU) >  db.t3.medium (4 GB RAM, 2 VCPU) > db.t3.large (8 GB RAM, 2 VCPU), > db.t3.xlarge (16 GB RAM, 4 VCPU).
  • With a DB instance especially, you are likely to be keeping it around for a while. This makes buying reserved instances (where you pay up front) very appealing. You can save upward of 40% this way.

Benefits

  • You can schedule backups and minor DB patch releases automatically on AWS. With Exadata Express, you had to manually run an export. You can even set your backup retention period.
  • You can perform point in time DB recovery.
  • You have visibility to metrics and can create automated alarms to monitor the health of your database.
  • You can configure your database with multi-availability zone redundancy.
  • 18C of the Oracle database is available.
  • You can perform button push DB upgrades in place. We can attest to this as we recently upgraded our new environment from 12C to 18C.
  • You can install any APEX version you want, important if your App is tied to a specific APEX version.
  • RDS has an integration with S3 (object storage) so you can copy files between the DB server and S3. This is great for ad-hoc data pump backups or moving schemas between DB instances.
  • You have access to the database file system via UTL_FILE.
  • You can create your own database wallets. This is especially important for a cloud environment which may need to use custom SSL certificates to gain entry into your company’s firewall.
  • Using other AWS services becomes much easier. For example, we run Tomcat on HTTP and have an AWS Application Load Balancer exposed to the internet. AWS provides SSL certificates for free when using a load balancer. If you have access to your domain DNS, you can easily create a friendly URL for you APEX environment e.g. https://apps.abccorp.com
  • If you need to send emails to customers from APEX that do not end up in their spam folder then AWS SES (Simple Email Service) is just a few clicks away. It creates an SMTP server which you can add to your APEX Internal Workspace and then any email you send from APEX gets sent out using SES. SES emails include the necessary DKIM information so that your email does not end up in your recipient’s spam folder (not something that could be said for Exadata Express).
  • Once you have a load balancer in place, you could even load balance between two ORDS servers so that you have increased redundancy and scalability.
  • You get a full ORDS install. This means you get to set the ORDS parameters the way you want them. This includes setting the JDBC connection pool, configuring Prehook, preprocess, postprocess, restEnabledSql and pagination maxRows. There is no way this is going to be available to you on the Autonomous Database Cloud Service.

Considerations

There is no perfect solution and there are certainly some things you should consider when moving to AWS.
  • Although the database is almost fully managed by AWS, you are on your own with regard to the ORDS server. You need to standup an EC2 server and install ORDS, Tomcat, setup your load balancer etc. This means it is not as hands off as Autonomous Database Cloud Service.
  • At time of writing, there is no way to install APEX PSE Bundle Patches to an RDS instance. Depending on the bug, this could be a big deal.
  • For license included versions of the Oracle Database, Standard Edition 2 is way more affordable than Enterprise Edition. If you have specific use cases that require Enterprise Edition features, then keep an eye on your costs.
  • Although AWS takes care of patching the DB server, you will be responsible for patching the EC2 server and upgrading Java, Tomcat and ORDS.

Conclusion

APEX RDS Conclusion
​I am not saying RDS is a magic bullet that allows you to achieve the hands-off capabilities of the Autonomous Database Cloud Service at a fraction of the cost. If you have the budget and you want absolutely nothing to do with the underlying architecture, then Autonomous Database Cloud Service is for you. If you want lower level access to the database (and / or ORDS) but want to be absolutely sure the database itself is being backed up and patched for you then it offers a great alternative. If you throw in easy access to other AWS services, this soon becomes a very compelling solution.
 
After all, for developers, the health and welfare of the database is really the scariest part of running your own RAD stack. If AWS is essentially taking care of this for you, the rest is reasonably straight forward.

Author

Jon Dixon, Co-Founder JMJ Cloud

0 Comments

Accessing AWS S3 from APEX with API Gateway & Lambda

10/4/2018

2 Comments

 

Introduction

Simplfying Access to AWS S3 from APEX Using API Gateway and Lambda
​Over the past 6 months, APEX has been the foundation for several innovative solutions provided by JMJ Cloud. Our most recent project leads us to the world of Amazon Web Services (AWS) with API Gateway, Lambda (serverless/function as a service) and S3 object storage. In this post, I will show you how we built a cost-effective solution to simplify posting files to S3 from APEX using API Gateway and Lambda.

​The Requirement

A few months ago, JMJ Cloud built a native mobile iOS App for a major manufacturing client
(Mobilize your Oracle Database with ORDS). Amongst other things, the App allowed shop floor workers to take pictures of images throughout the manufacturing process and post them to Amazon S3 storage. In addition to the iOS App, we built a large APEX application (integrated with Oracle EBS) for front office staff to manage jobs in the plant. The requirement that lead me to this blog post is to provide the ability for the desktop APEX users to post documents and files to S3.

The Challenge

AWS does not have an SDK for S3 and PL/SQL (no surprises there). This makes managing objects in S3 from APEX and or PL/SQL a major challenge. The number of steps AWS make you go through to call their S3 web services are overwhelming and if you are on R11 of the database (with no dbms_crypto to create a SHA-256 hash) it is almost impossible to achieve. Some folks have tried to solve this problem through open source solutions such as Alexandria (link). The problem is that as Amazon evolves its S3 offering, the open source solutions aren’t always able to keep up. We needed a way to simplify the interface with S3 and use this from multiple clients including APEX.

​The Solution

The solution to simplify the process of interfacing with S3 involves two other AWS products, API Gateway and Lambda. In the diagram below, I am illustrating how we use these technologies to get a pre-signed S3 URL. With a pre-signed URL, you can perform an action on a specific S3 object with just the URL (no other keys or hashing required). We can also use a similar approach to directly post files to S3 but both API Gateway and Lambda have payload size limits. API gateway has a limit of 10mb and Lambda a limit of 6mb. By the time you account for base64 encoding the files, this gives you a max file size of about 4mb.
Picture
Let’s step through the diagram to see what is happening:

1 – Calling the API Gateway Service
In APEX we use APEX_WEB_SERVICE to call a simple REST service end point on API Gateway. A small JSON document is constructed with information about the file, bucket and URL expiration. You can include Tags for the S3 file in your headers when posting the file. This will ensure those tags are included in the Pre-signed URL and attached to the S3 object when you come to POST it.

2 – The API Gateway Web Service
The API Gateway end point is secured using a key which is maintained in API Gateway. There are other alternatives (such as  AWS Cognito). Because we are storing the key securely in our Oracle EBS database, using a key is secure enough. As well as securing the service, API Gateway provides additional functionality such as validating the JSON payload. This avoids un-necessary calls to the Lambda when invalid payloads are passed. Other useful features include built in monitoring and throttling to name a few.

The end point for our API Gateway function is a Lambda function. AWS has a tight integration between these two technologies so hooking them up is pretty straight forward.

3 – The Lambda Function
Serverless computing allows you to build and run applications and services without thinking about servers. With serverless computing, your application still runs on servers, but all the server management is done by AWS. At the core of serverless computing is AWS Lambda, which lets you run your code without provisioning or managing servers.

In certain scenarios, serverless technologies such as Lambda offer significant advantages over traditional approaches. It is always available, you only pay for it when you invoke the function and it scales automatically. You can develop Lambda functions in Node, Java, Python, C# and Go. No news on when PL/SQL will be supported :)

We built our function in Node and it has just 30 lines of code. The function imports the AWS S3 tools library, parses the JSON payload passed in by API Gateway and calls the S3 API to get the pre-signed URL. Again, the tight integration between Lambda and S3 comes in handy, the call to the S3 API is extremely simple.

4 – The Response
When the Lambda function is done, it passes the pre-signed URL back to API Gateway within a JSON document. API Gateway then passes the JSON back to our request in PL/SQL.

5 – Posting the File
Now that we have a pre-signed URL we can post our file directly to S3 without the need for keys and hashing. We again use APEX_WEB_SERVICE to post our BLOB to S3. 

PL/SQL Code

The PL/SQL code to call the API Gateway Service to get the pre-signed URL and then post a file to that URL is about 40 lines of code. Add the 30 lines of code for the Lambda function and we are still under 100 lines of code. Not too shabby!
 
Getting the Pre-Signed URL (includes adding tags)

    

Considerations

  • ​Indexing Files:
    • Even though S3 does allow for tagging, it is fundamentally object storage and you are responsible for indexing the file in your database. In view of this, whenever you post a file to S3 you should also be adding a record to a table in your local database, so you can find it again later.
    • Having said that, you should also tag your files in the event your index fails for some reason.
  • Security:
    • Because we have a pre-signed URL, anyone with that URL can update the associated object in S3. In view of this, be sure to limit the expiration time of the URL to just the amount of time it takes to POST the file. (think minutes not hours).
  • JavaScript:
    • You can also use this approach to post files directly from JavaScript as opposed to using a ‘File Browse’ item and processing the file in PL/SQL. You can use a PL/SQL dynamic action to get the pre-signed URL then post the file directly from JavaScript on your APEX page which avoids a round trip to the database.
  • Wallets:
    • Amazon has different SSL certificates for API Gateway and S3 which means you will need to load both certificates into your database wallet.
  • Lambda Warmup:
    • You pay for each execution of your Lambda function. When your code is inert you don’t pay anything for it. In view of this, AWS only spins up a container for your function when it is executed. This can lead to a lag of a couple of seconds the first time you call the function after a period of inactivity. This is not a big deal for this use case but is worth understanding.

Conclusion

​The case for serverless technologies is building as their capabilities continue to grow. In this post, I have described a relatively simple use case. I hope it has given you some ideas of how this technology can help you to solve similar problems.

Author

Jon Dixon, Co-Founder of JMJ Cloud

2 Comments

    RSS Feed

    Popular Posts

    - APEX Dog Food
    - Cloud ERP & APEX Mashup
    - Modernizing EBS with APEX
    - Easy APEX_WEB_SERVICE
    - Running APEX in RDS
    - ORDS What & Why?

    Categories

    All
    APEX
    AWS
    Fusion Cloud ERP
    INTEGRATION
    OCI
    ORDS
    PaaS
    RAD
    REST
    SOAP

    Archives

    January 2021
    October 2020
    September 2020
    June 2020
    May 2020
    April 2020
    February 2020
    January 2020
    October 2019
    September 2019
    August 2019
    July 2019
    June 2019
    April 2019
    March 2019
    February 2019
    January 2019
    December 2018
    November 2018
    October 2018
    September 2018
    August 2018
    July 2018
    June 2018
    May 2018
    April 2018
    March 2018
    February 2018
    January 2018
    September 2017
    August 2017
    July 2017
    June 2017
    February 2017
    January 2017
    December 2016
    November 2016
    October 2016
    September 2016
    August 2016
    July 2016

Company

About
Contact
Blog
  • Home
  • Projects
  • Blog
  • About Us
  • Contact Us