JMJ CLOUD
  • Home
  • Projects
  • Blog
  • About Us
  • Contact Us

Our Blog

RIP Exadata Express, where do I run my RAD stack now?

9/12/2019

0 Comments

 

Introduction

APEX Hosting on AWS RDS
​As you may be aware, Exadata Express is coming to the end of its life. If you like us have found this platform both useful and affordable, then you will be wanting to know what’s next? Where in the cloud can I deploy my RAD (REST, APEX, Database) stack now? In this post I will explore an alternative which we have implemented here at JMJ Cloud.

Background

​We have been using Exadata Express and JMJ Cloud for more than 2 years. During that time, we helped five of our clients implement Exadata Express. Use cases ranged from custom cloud point solutions to ERP Cloud PaaS extensions. I have also blogged about this platform on a number of occasions. Exadata Express offered a highly functional PaaS solution for as little as $175 a month. We (and our customers) were very happy with it. In fact, I thought this product was a sign that Oracle had finally turned the corner to becoming a major cloud player. It provided a very low barrier to entry with more feature rich alternatives ready and waiting when your business grew into them. This is essentially how AWS became so successful. One day you start with half a dozen EC2 servers for a couple of hundred dollars a month and a few years later you are spending a million dollars a month (think Netflix). Oracle on the other hand typically starts by charging a lot and then work their way up from there.

How did we Get Here?

A month or so ago, our Salesperson emailed us… Here is where I need to pause for a small rant. Why on earth, if you are selling automated cloud solutions should I ever have to deal with a Salesperson? I want a server; I know what size server I want, and I know how much I want to pay for it. From my perspective, having a Salesperson in the middle of this is expensive from Oracle’s point of view and extremely frustrating from my point of view. OK, back to the story… Our Salesperson reached out and said we could not renew our Exadata Express contract and oh, by the way, you have a month to get off the platform. After a little discussion (i.e. an hour of my time on the phone) we converted that 1 month into 3 months. We bought ourselves some runway, but we still needed another platform. Enter the Salesperson (again). After another one hour of my time he proceeded to explain the wonders of the Autonomous Database Cloud Service. Don’t get me wrong, this service shows a lot of promise and it is going to be great for our larger customers. At more than $1,200 per month, however, it is not going to fly for us or our smaller customers wanting to run the RAD stack in the cloud. It is nearly 7 times more expensive than Exadata Express. If you wanted to use Autonomous Database with APEX for a Cloud ERP extension and you need a DEV, TEST and PROD instance that is $3,600+ per month instead of $600 / month.
 
To add insult to injury, there is not even a button push migration path from Exadata Express to Autonomous Cloud. This means that even if you have the money to pay for Autonomous Cloud, you are going to have to fork more money to migrate.

The Alternative

APEX Hosting the Alternative
I would love nothing more than to say I whole heartedly recommend folks move to the Autonomous Database Cloud Service but that is not where I am going. The answer to our problem was in a service we have dabbled with before which is Amazon’s RDS (relational database service) service along with an EC2 (elastic compute) server running ORDS. I have never really been completely happy recommending this approach for the RAD (REST, APEX, Database) stack before, two things have changed to make this a much more appealing approach.
  • You can now perform a full ORDS install on an RDS instance. This is thanks to new functionality in ORDS 19.1 which allows you to run the install as a user other than SYS. This is especially important now that REST services are no longer available in APEX alone.
  • AWS have started making the latest version of ORDS available on RDS much sooner than they used to. For example, as of right now, September 2019, you can run ORDS 19.2 and APEX 19.1 on RDS, whereas Exadata Express is running ORDS 3 (ouch) and APEX 18.2.

Architecture

​The diagram below shows the architecture we are using at JMJ cloud.
Picture

Cost Breakdown

Here is a cost breakdown for the above architecture.
APEX AWS Cost Breakdown
Notes
  • We did not go for provisioned IOPS (input output per second, A measure of disk speed) which means we get burst-able IOPS. This is fine for short bursts of activity but if you expect prolonged volume activity, you will want to pay for provisioned IOPS which will increase the cost.
  • RDS service pretty much doubles as you go from db.t3.small (2GB RAM, 2 VCPU) >  db.t3.medium (4 GB RAM, 2 VCPU) > db.t3.large (8 GB RAM, 2 VCPU), > db.t3.xlarge (16 GB RAM, 4 VCPU).
  • With a DB instance especially, you are likely to be keeping it around for a while. This makes buying reserved instances (where you pay up front) very appealing. You can save upward of 40% this way.

Benefits

  • You can schedule backups and minor DB patch releases automatically on AWS. With Exadata Express, you had to manually run an export. You can even set your backup retention period.
  • You can perform point in time DB recovery.
  • You have visibility to metrics and can create automated alarms to monitor the health of your database.
  • You can configure your database with multi-availability zone redundancy.
  • 18C of the Oracle database is available.
  • You can perform button push DB upgrades in place. We can attest to this as we recently upgraded our new environment from 12C to 18C.
  • You can install any APEX version you want, important if your App is tied to a specific APEX version.
  • RDS has an integration with S3 (object storage) so you can copy files between the DB server and S3. This is great for ad-hoc data pump backups or moving schemas between DB instances.
  • You have access to the database file system via UTL_FILE.
  • You can create your own database wallets. This is especially important for a cloud environment which may need to use custom SSL certificates to gain entry into your company’s firewall.
  • Using other AWS services becomes much easier. For example, we run Tomcat on HTTP and have an AWS Application Load Balancer exposed to the internet. AWS provides SSL certificates for free when using a load balancer. If you have access to your domain DNS, you can easily create a friendly URL for you APEX environment e.g. https://apps.abccorp.com
  • If you need to send emails to customers from APEX that do not end up in their spam folder then AWS SES (Simple Email Service) is just a few clicks away. It creates an SMTP server which you can add to your APEX Internal Workspace and then any email you send from APEX gets sent out using SES. SES emails include the necessary DKIM information so that your email does not end up in your recipient’s spam folder (not something that could be said for Exadata Express).
  • Once you have a load balancer in place, you could even load balance between two ORDS servers so that you have increased redundancy and scalability.
  • You get a full ORDS install. This means you get to set the ORDS parameters the way you want them. This includes setting the JDBC connection pool, configuring Prehook, preprocess, postprocess, restEnabledSql and pagination maxRows. There is no way this is going to be available to you on the Autonomous Database Cloud Service.

Considerations

There is no perfect solution and there are certainly some things you should consider when moving to AWS.
  • Although the database is almost fully managed by AWS, you are on your own with regard to the ORDS server. You need to standup an EC2 server and install ORDS, Tomcat, setup your load balancer etc. This means it is not as hands off as Autonomous Database Cloud Service.
  • At time of writing, there is no way to install APEX PSE Bundle Patches to an RDS instance. Depending on the bug, this could be a big deal.
  • For license included versions of the Oracle Database, Standard Edition 2 is way more affordable than Enterprise Edition. If you have specific use cases that require Enterprise Edition features, then keep an eye on your costs.
  • Although AWS takes care of patching the DB server, you will be responsible for patching the EC2 server and upgrading Java, Tomcat and ORDS.

Conclusion

APEX RDS Conclusion
​I am not saying RDS is a magic bullet that allows you to achieve the hands-off capabilities of the Autonomous Database Cloud Service at a fraction of the cost. If you have the budget and you want absolutely nothing to do with the underlying architecture, then Autonomous Database Cloud Service is for you. If you want lower level access to the database (and / or ORDS) but want to be absolutely sure the database itself is being backed up and patched for you then it offers a great alternative. If you throw in easy access to other AWS services, this soon becomes a very compelling solution.
 
After all, for developers, the health and welfare of the database is really the scariest part of running your own RAD stack. If AWS is essentially taking care of this for you, the rest is reasonably straight forward.

Author

Jon Dixon, Co-Founder JMJ Cloud

0 Comments

SMS Messaging from Oracle ERP with ORDS and Twillio

9/6/2018

3 Comments

 

Introduction

SMS Messaging from Oracle ERP with ORDS and Twillio
​Twillio has become the gold standard for automating the sending and receiving of SMS messages. They have a well-documented and easy to use API, with features designed to make sending and managing SMS messages easy. In our latest engagement, JMJ Cloud is using Twillio to help our customer provide important and timely information (from Oracle e-Business Suite) to their employees via SMS. In this post I will review the requirement and the architecture we put in place to make it all happen.

​The Requirement

​Our customer needs to notify employees when certifications they hold are approaching their expiration date. Details of the employee’s certifications are maintained in Oracle e-Business Suite (EBS) on their HR record. The nature of their work dictates that these employees cannot perform their job without a current certificate. The work takes employees all over the world and being able to reach them when they are out of the office made SMS a great choice.
 
The customer also has plans to send a number of other SMS messages from EBS and other internal systems. Building a generic outbound and inbound queuing process was necessary to facilitate re-use of the solution by these other sources.

A Little Bit about Twillio

Twillio Logo
​If you have used Lyft, Airbnb, or Netflix, you have probably used Twilio. In fact, Twilio powers communications for over 40,000 businesses around the world. Simply put, Twilio is a developer platform for communications. Software teams use Twilio APIs to add capabilities like voice, video, and messaging to their applications.

There are a couple of things you need to setup in Twillio before you can start sending SMS messages:
  • Signup for an account.
  • Purchase a number. This is the outbound phone number that will appear on message recipients’ phones. It is also the number Twillio is watching to handle responses to SMS messages. Numbers range in price from $1 / month to thousands for shortened numbers. You do get to choose the number you want to use from a list of available numbers. The process is a little like choosing a cell phone number. You may end up needing multiple numbers based on your requirements, especially if you need to handle SMS responses differently for each requirement.
  • Configure a Web Hook to receive responses. For each number you buy, you can configure a URL which Twillio will POST to whenever someone responds to a text message sent from that number. This web hook will post the number the response was sent from as well as the response message itself. As you will see from the architecture diagram below, we chose to configure an ORDS REST service to receive these responses.

​Architecture

Picture

​Outbound Flow

  1. The outbound flow starts with a feeder system (in our case Oracle HR) adding an SMS to the outbound queue table. Each feeder system is categorized by a ‘channel’ column on the table. The table also stores the SMS message, the to and from number, key identifying information for the recipient (name, internal id etc.) and status columns.
  2. A scheduled Outbound Queue processing concurrent program runs every hour (during the day). This program performs the following actions:
    1. Calls the Twillio SMS API to Queue the message with Twillio.
    2. Calls the Twillio message status API to confirm previously queued messages were actually delivered.
    3. Logs Twilio’s unique message id to the outbound queue table.
    4. Logs any errors (e.g. invalid mobile number) to the outbound queue table
    5. Sends an email with failed deliveries to all interested parties.
  3. When Twillio receives a request, it will:
    1. Check to see if the number the SMS is being sent to has opted out of receiving SMS messages from your sending number. If it has then it will return an error message and not send the SMS.
    2. Perform basic validations on the recipients’ number and return an error if applicable.
    3. If there are no errors, it will queue the message to be sent.

Note
: Twillio does not send the SMS synchronously. Just because you get a success message back from the send API, does not mean the message was delivered. You need to call a second message status API to confirm successful delivery.

​Response / Inbound Flow

  1. The inbound flow starts with an employee responding to an SMS. Possible reasons for responding to an SMS are:
    1. Request to opt out of future SMS messages.
    2. Response to a question as part of a ‘conversation’ with one of your systems.
  2. When Twillio receives the response, it does the following:
    1. If the user responds with an opt out message e.g. ‘STOP’ then it will tag that number as blocked and prevent you from sending further messages to that number until they opt back in.
    2. Formats the message and information about the responder into a JSON document and posts the response to the web hook you configured (ORDS POST Service).
  3. The ORDS POST service adds the response to a queue table sitting in Exadata Express.
  4. A scheduled inbound queue Concurrent Program runs to pick up new responses from Exadata Express. It does this by calling another ORDS web service on Exadata Express which returns the responses. The Concurrent Program then copies the data to a local table in the EBS database.
  5. A final concurrent program runs which handles responses received in the local table. This includes activities such as updating the employees HR record based on their response. This program also sends out an email digest of all received SMS responses to the managers of the employees who have responded.

Monitoring
A. An APEX dashboard provides administrators the ability to monitor both the inbound and outbound queues and respond to errors etc.

​Considerations

This section focuses on some key considerations when implementing an SMS solution.

Data Cleansing and Managing Errors​

Picture
​Chances are, the phone numbers you have stored in your ERP system are not being used by an automated solution such as this. When a human sees a number like +11234567890, 1234567890 or (123)456-7890 they can figure out the number and the country code.

A computer system, however needs a more exact format to follow. It is also likely you will not have mobile numbers for everyone. Twillio requires that you send the country code with each SMS message.  All of this means you will have to work at collecting mobile numbers for everyone and introduce a standard for storing these numbers that includes the international dialing code (e.g. +1 or +44). Twillio can help with the data cleansing aspect by providing APIs that you can call to validate a number.
 
As well as an initial data cleansing effort, you will need to provide mechanisms to make managers aware when SMS messages fail to get sent due to invalid numbers. A business process also needs to be put in place to correct or add numbers to keep the process flowing smoothly.

Handling SMS Conversation

Chat
​Even though Twillio POSTs all the data you need to your web hook, there is no way of knowing which piece of an ongoing thread the user is responding to. If you look at the text history between you and a friend, you will often see responses out of sequence with the questions. Each interaction on an SMS thread does not have an identifier to help you keep a specific conversation together. In view of this, your back-end processing logic will need to be smart and look for key words and context clues. You also need to be very specific in the message you send out to make sure the user knows exactly how to respond.​

Using Different From Numbers

If you have different use cases for sending text messages and you need to handle responses to messages differently, then you may want to consider buying more than one outbound number. Having multiple numbers will make it easier to know the context of a response and handle processing of it more accurately.
 
One other thing to note is that the recipient is not going to recognize the number the SMS is being sent from. The first 15-20 characters of your SMS needs to clearly indicate who it is from and its urgency. For example, start your message with ‘From: ABC INC…’​

When to Send SMS Message

Angry Emoji
​Receiving timely information can make a huge impact. Having said that, SMS messages are a far more intrusive form of communication than email. You should not use it as the go-to method of communication (especially with employees). Use it sparingly and only when it really adds value.

​CAUTION:  do not schedule outbound messages to be sent in the middle of the night. Be careful when scheduling outbound messages, that you don’t send them in the middle of the night. The only thing more annoying than a text from your boss is one that is sent at midnight when your boss is sound asleep!

Opting Out​

Opt Out
​​Opting out of receiving SMS messages is an important part of the end-to-end flow. Twillio handles opting out very well but you need to provide instructions to the recipient on how they can opt out (and opt back in). You will also need to make managers aware of those people who have opted out.

Conclusion

Sending automated SMS messages is by no means a new thing. The technology and services that are available today, however, are making it far easier to do. Used correctly, SMS can provide the information your employees or customers need right when they need it.
 
Yet again, the RAD stack provides many of the technologies you need to integrate modern and innovative technology solutions such as Twillio into your existing ERP. One more reason to install APEX and ORDS today!
3 Comments

    RSS Feed

    Popular Posts

    - APEX Dog Food
    - Cloud ERP & APEX Mashup
    - Modernizing EBS with APEX
    - Easy APEX_WEB_SERVICE
    - Running APEX in RDS
    - ORDS What & Why?

    Categories

    All
    APEX
    AWS
    Fusion Cloud ERP
    INTEGRATION
    OCI
    ORDS
    PaaS
    RAD
    REST
    SOAP

    Archives

    January 2021
    October 2020
    September 2020
    June 2020
    May 2020
    April 2020
    February 2020
    January 2020
    October 2019
    September 2019
    August 2019
    July 2019
    June 2019
    April 2019
    March 2019
    February 2019
    January 2019
    December 2018
    November 2018
    October 2018
    September 2018
    August 2018
    July 2018
    June 2018
    May 2018
    April 2018
    March 2018
    February 2018
    January 2018
    September 2017
    August 2017
    July 2017
    June 2017
    February 2017
    January 2017
    December 2016
    November 2016
    October 2016
    September 2016
    August 2016
    July 2016

Company

About
Contact
Blog
  • Home
  • Projects
  • Blog
  • About Us
  • Contact Us