​With the evolution of ERP Cloud and Oracle Cloud Infrastructure (OCI), we are starting to see ERP Cloud utilize OCI features. This is a positive move and it will allow ERP Cloud customers to gain more value from their ERP Cloud investment. One area where we have seen this partnership blossom is with ERP Cloud BI Cloud Connector (BICC) and OCI Object Storage. In this post I will cover how you can leverage more from this integration using OCI Events and Notifications.

​Oracle BI Cloud Connector (BICC) provides a robust mechanism for performing bulk exports of data from ERP Cloud. Exported CSV files can be directed to Universal Content Management (UCM) (contained within the ERP Cloud environment), or to an OCI Object Storage Bucket. BICC exports are scheduled through the ERP Cloud Enterprise Scheduler Service (ESS). For those of you familiar with e-Business Suite, this is like the Concurrent Manager. So, how do we know when a scheduled job has finished so we can go fetch the file? This is the problem we will address in this post.
 
Although the example below describes a scenario using ERP Cloud, this pattern can be re-used for any scenario where you need to perform an action when a file is created in an OCI bucket, but you don’t know when the file will be created.

Oracle Cloud Infrastructure includes two integration tools that can help us with this problem. These are Events and Notifications.
 
Events Service
Events fire when certain things happen on your OCI infrastructure. In our case we want to launch an event whenever a file is created in a specific OCI Object Storage bucket. An event on its own, however, is not very useful. We need the Event to do something and this is where Notifications come in.
 
Notifications Service
A Notification can perform one of the following operations:

• Email
• Function (Cloud Based Function e.g. Java)
• HTTPS (Custom URL)
• PagerDuty
• Slack

 
The key notification for us is ‘HTTPS (Custom URL)’. This notification type is further described in section 3 below. When this type of Notification fires, it posts a JSON payload to the HTTPS Endpoint that you define. The content of the payload is specific to the OCI service that causes the Event to fire. In our case, it will contain information about the file that was created.

1 - File from ERP Cloud
In our example this file is coming from BI Cloud Connector (BICC) on ERP Cloud. As I mentioned earlier, BICC allows you to perform high volume extracts of data from ERP Cloud to either local Universal Content Manager (UCM) or an OCI Object Store Bucket. We will be using OCI Object store as this gives us more options and easier access to the files once created. BICC allows you to either call a web service to launch an extract job or to use ERP Cloud to schedule the extract. In view of this, files could appear in our OCI Object Store Bucket at any time. We need to know when the file is created so we can go and fetch it.
 
Object Store Bucket
You will need to create an Object Store bucket to receive your file. When creating your Object Store bucket, be sure the ‘Emit Object Events’ option is checked. This is what allows us to subscribe to events on the bucket.

2 - Event
As soon as the file is created in our object store bucket, the event fires. Events consist of an Event Type (this describes the action that should trigger the event) and an Event Action (what you want to happen when the event occurs). In our case the event type is 'Object - Create' and the Event Action is to call a Notification.

3 - Notification
​The Notification is the operation we want to perform when our event fires. We are using the ‘HTTPS (Custom URL)’ operation. Notifications consist of Topics which allow us to categorize our notifications. For each Topic, we can create multiple Subscriptions. This allows us to let multiple systems know when an Event occurs. In this example, we are referencing our ORDS Endpoint.

If your ORDS end point is unavailable when it is called by the Notification, OCI will re-try to post to your service for a period of up to 2 hours.

​The Endpoint service needs to have a POST handler to accept the payload from the Notification. In our case, our Endpoint is an ORDS REST service running on an APEX environment (see step 4 for details).

When you first create a notification, it will go into a status of ‘Pending confirmation’.

At the time you create the Notification, OCI sends a payload to your Endpoint containing a confirmation URL. The confirmation step ensures that you have control over the Endpoint that you define in your Notification.

4 – ORDS Service
The ORDS web service definition is pretty simple. Here is a screen shot (from SQLDeveloper) of the Module, Template and Handler for a service that can receive a payload from an OCI Notification.

The PL/SQL code you add to the handler can do anything you want. In our case it submits a Database Scheduler Job to fetch and process the file asynchronously.
 
5 – Process File
In our DB Scheduler Job, we use the OCI Object Store REST services to fetch the file from the OCI Object Store bucket. This connectivity between APEX and OCI was made much easier in APEX 19.2 when Web Credentials were enhanced to include the ‘Oracle Cloud Infrastructure (OCI)’ Authentication Type. This allows us to configure a native Web Credential for connecting to OCI Web Services. I recommend this Oracle A-Team Blog which describes in detail the OCI and APEX setups that are required to get this integration working.
 
After you have fetched the file (using APEX_WEB_SERVICE), you can then parse it (using APEX_DATA_PARSER) and process it as necessary, making the data available for your APEX Applications.

​I have been using APEX and ORDS on Amazon Web Services (AWS) for a number of years and until recently, I was not sure Oracle was ever going to compete. Oracle Cloud Infrastructure has come a long way in the past 12 months. While it still falls far short of AWS in terms of functionality, the functionality that is there is intuitive and works well. If you add great initiatives like ‘Always Free’ ATP, ORDS and APEX then you have a platform that is worth getting to know..
 
As APEX/ORDS developers and architects I encourage you to explore OCI. I also think we should start  to think differently about how we build our APEX solutions to utilize OCI where it makes sense. 

​Since APEX 18.1, there has been a new way to call OAuth2 Client Credentials secured web services using APEX_WEB_SERVICE. This new method alleviates the need for developers to store OAuth2 credentials, manage token expiration and also has potential performance benefits. In this post I will describe the new approach and its benefits.

In the OAuth2 client credentials flow you must first call a token web service, passing a client ID and client secret. If these credentials are valid, the token service will then pass back a token. The token can then be used to call certain secured web services covered by the token. The token returned by an OAuth2 client credentials service looks something like the below.

I must confess that I can get stuck in my ways. It often takes a weekend and a few hours buried in the APEX API guide for me to get caught up with the latest and greatest. Case in point, I have been using APEX_WEB_SERVICE for several years and most of my code to call web services secured by Ouath2 Client Credentials looks like this:

1. Fetch token service URL, Client ID and Client Secret from a custom table.
2. Call the OAuth token service to get token using APEX_WEB_SERVICE.
3. Optionally store the token for use again (within its expiration window).
4. Call the main web service using APEX_WEB_SERVICE, passing the token.
5. Repeat as many times as the user calls the web service.

 
Don’t get me wrong, this is still much more convenient than using UTL_HTTP. As a developer, however, I am still having to manage the following:

1. Storing the token end point and the client credentials (most likely not very securely).
   
2. Deciding when the token expires and fetching another token (more likely not bothering to do this and just getting a new token every time I call the main web service).

Of course, there are drawback to this.
 
Storing the Credentials
​I have to build an APEX page to store the token URL and credentials and also some APIs to set and get these values. I then have to build APIs to manage token expiration and decide when to get a new token. Finally, I have to maintain this code and nurse it through upgrades etc. All this was fun the first time but I would much rather have APEX handle this for me so I can focus on delivering business logic.
 
Securing the Credentials
This approach also introduces the obvious security concern which is that I may not be encrypting the credentials when I store them.
 
Performance
Unless you build code to store the expiration time for the token, you are most likely going to call the token service every time you call the main web service. While token services are usually pretty light weight, you are incurring an extra round trip from your database to the endpoint every time you call the main web service. This can add up to a second to every web service call which is noticeable to your end users.

Ever since the APEX development team introduced APEX 18.1 you could sense a shift toward making it easier for developers to talk to the outside world using APEX. This all hinges around the APEX_EXEC package and the drive toward making every APEX component equally as happy sourcing it’s data from a web service as it is from a SQL statement on the local database.
 
Since APEX 18.1 we can make calls to APEX_WEB_SERVICE without having to deal with first calling the associated token service or having to manage token expiration. This is made possible with the addition of 2 parameters p_credential_static_id and p_token_url. These parameters allow you to pass a reference to a set of ‘Web Credentials’ and the token URL to APEX_WEB_SERVICE and it will handle the REST (pun intended).
 
APEX_EXEC For all use cases that fit, I strongly encourage you to look into APEX_EXEC, instead of APEX_WEB_SERVICE. APEX_EXEC takes even more of the load off the developer and has a number of other advantages which I don’t have time to get into in this post. Carsten Czarski wrote a great overview in this post.

​As always, examples speak a thousand words. Before we can reference a set of Web Credentials in our code, we need to define them in APEX. Web Credentials are stored at the Workspace level. Once logged into your APEX Workspace, navigate to App Builder > Workspace Utilities > All Workspace Utilities > Web Credentials.
 
In addition to creating and editing Web Credentials in the APEX UI, you can also set them programmatically using the APEX_CREDENTIAL API (available since APEX 18.2). This API has a number of uses, one of which is to help when migrating new credentials to TEST and PROD instances.

​Note: When you edit Web Credentials APEX does not show you the client secret. It is there, you just can’t see them (for security reasons).

I now want to look back on the three drawbacks from the old five steps approach and see how the two extra parameters above have helped me. The main benefit of course is that we now have just two steps instead of the 5 steps in the original approach:

1. Call the main web service using APEX_WEB_SERVICE, passing Web Credentials Static ID and Token Web Service URL.
2. Repeat as many times as the user calls the web service (APEX handles fetching new tokens etc.).

 
Storing the Credentials
The OAuth2 Client Credentials can now be stored natively in APEX using ‘Web Credentials’. APEX provides a UI to manage these credentials and APIs to set these credentials programmatically.
 
Securing the Credentials
Using Web Credentials, even the Workspace Administrator is not able to view the client secret for a given client. In addition, although there are APIs to set the credentials there are none to get credentials. The only way to use them is via APEX APIs.
 
Performance
​When you pass values for p_credential_static_id and p_token_url you are essentially handing over management of the OAuth2 token to APEX. It will store the expiration for the token, and it will only get a new token when it needs to.
 
All this leaves you as a developer to focus on calling the main web service and parsing the results.

​The APEX development team continues to make it easier for APEX developers to work with REST based web services. This makes it easier and easier for developers to use cloud-based services like SMS, Address Verification, Machine Learning, Object Storage etc. etc. Having easy access to these cloud services allows developers to provide game changing features in their APEX apps.
 
Enhancement Request: It would be great if instead of having to pass p_token_url, we could pass the static id of the remote server plus a suffix for the token service end point. This would prevent us from having to store the URL of the token service in a custom table. Remote Servers are defined under ‘Remote Servers’ in the Workspace Utilities area (right above Web Credentials).

​As you may be aware, Exadata Express is coming to the end of its life. If you like us have found this platform both useful and affordable, then you will be wanting to know what’s next? Where in the cloud can I deploy my RAD (REST, APEX, Database) stack now? In this post I will explore an alternative which we have implemented here at JMJ Cloud.

​We have been using Exadata Express and JMJ Cloud for more than 2 years. During that time, we helped five of our clients implement Exadata Express. Use cases ranged from custom cloud point solutions to ERP Cloud PaaS extensions. I have also blogged about this platform on a number of occasions. Exadata Express offered a highly functional PaaS solution for as little as $175 a month. We (and our customers) were very happy with it. In fact, I thought this product was a sign that Oracle had finally turned the corner to becoming a major cloud player. It provided a very low barrier to entry with more feature rich alternatives ready and waiting when your business grew into them. This is essentially how AWS became so successful. One day you start with half a dozen EC2 servers for a couple of hundred dollars a month and a few years later you are spending a million dollars a month (think Netflix). Oracle on the other hand typically starts by charging a lot and then work their way up from there.

A month or so ago, our Salesperson emailed us… Here is where I need to pause for a small rant. Why on earth, if you are selling automated cloud solutions should I ever have to deal with a Salesperson? I want a server; I know what size server I want, and I know how much I want to pay for it. From my perspective, having a Salesperson in the middle of this is expensive from Oracle’s point of view and extremely frustrating from my point of view. OK, back to the story… Our Salesperson reached out and said we could not renew our Exadata Express contract and oh, by the way, you have a month to get off the platform. After a little discussion (i.e. an hour of my time on the phone) we converted that 1 month into 3 months. We bought ourselves some runway, but we still needed another platform. Enter the Salesperson (again). After another one hour of my time he proceeded to explain the wonders of the Autonomous Database Cloud Service. Don’t get me wrong, this service shows a lot of promise and it is going to be great for our larger customers. At more than $1,200 per month, however, it is not going to fly for us or our smaller customers wanting to run the RAD stack in the cloud. It is nearly 7 times more expensive than Exadata Express. If you wanted to use Autonomous Database with APEX for a Cloud ERP extension and you need a DEV, TEST and PROD instance that is $3,600+ per month instead of $600 / month.
 
To add insult to injury, there is not even a button push migration path from Exadata Express to Autonomous Cloud. This means that even if you have the money to pay for Autonomous Cloud, you are going to have to fork more money to migrate.

I would love nothing more than to say I whole heartedly recommend folks move to the Autonomous Database Cloud Service but that is not where I am going. The answer to our problem was in a service we have dabbled with before which is Amazon’s RDS (relational database service) service along with an EC2 (elastic compute) server running ORDS. I have never really been completely happy recommending this approach for the RAD (REST, APEX, Database) stack before, two things have changed to make this a much more appealing approach.

• You can now perform a full ORDS install on an RDS instance. This is thanks to new functionality in ORDS 19.1 which allows you to run the install as a user other than SYS. This is especially important now that REST services are no longer available in APEX alone.
• AWS have started making the latest version of ORDS available on RDS much sooner than they used to. For example, as of right now, September 2019, you can run ORDS 19.2 and APEX 19.1 on RDS, whereas Exadata Express is running ORDS 3 (ouch) and APEX 18.2.

​The diagram below shows the architecture we are using at JMJ cloud.

Here is a cost breakdown for the above architecture.

There is no perfect solution and there are certainly some things you should consider when moving to AWS.

• Although the database is almost fully managed by AWS, you are on your own with regard to the ORDS server. You need to standup an EC2 server and install ORDS, Tomcat, setup your load balancer etc. This means it is not as hands off as Autonomous Database Cloud Service.
• At time of writing, there is no way to install APEX PSE Bundle Patches to an RDS instance. Depending on the bug, this could be a big deal.
• For license included versions of the Oracle Database, Standard Edition 2 is way more affordable than Enterprise Edition. If you have specific use cases that require Enterprise Edition features, then keep an eye on your costs.
• Although AWS takes care of patching the DB server, you will be responsible for patching the EC2 server and upgrading Java, Tomcat and ORDS.

​I am not saying RDS is a magic bullet that allows you to achieve the hands-off capabilities of the Autonomous Database Cloud Service at a fraction of the cost. If you have the budget and you want absolutely nothing to do with the underlying architecture, then Autonomous Database Cloud Service is for you. If you want lower level access to the database (and / or ORDS) but want to be absolutely sure the database itself is being backed up and patched for you then it offers a great alternative. If you throw in easy access to other AWS services, this soon becomes a very compelling solution.
 
After all, for developers, the health and welfare of the database is really the scariest part of running your own RAD stack. If AWS is essentially taking care of this for you, the rest is reasonably straight forward.