Infrastructure at your Service

Mehdi Bada

Managing Oracle Big Data Cloud – CE with REST API

By October 19, 2017 Big Data, Cloud, Oracle 3 Comments

In this blog post, we will see how to manage Oracle Public Cloud Big Data service Compute Edition with REST API. Scheduling the start/stop/restart of a metered PaaS in the Oracle Cloud can be interesting for managing efficiently the consumption of your cloud credits.

We should first have a look at the official documentation so as to understand what the API is composed of. https://docs.oracle.com/en/cloud/paas/big-data-compute-cloud/csbdp/QuickStart.html 

Use the following URL composition to get access to REST endpoint:
https://region-prefix.oraclecloud.com/resource-path

According to Oracle documentation, the following information should be taken into account.

Connection Information:

  • Identity Domain: axxxxxx
  • REstFull URL: https://psm.europe.oraclecloud.com/
  • username -password

Terminology:

  • {instanceName} = Name of the BDCS-CE service (= Cluster Name)
  • {identityDomainId} = “X-ID-TENANT-NAME: axxxxxx”
  • {function} = start, stop, restart
  • {allServiceHosts} = the entire cluster VMs (all instances which composed the cluster)
  • “Accept: <value>” = Media Type (default value = application/json)

Before starting an automation script to manage your Big Data cluster, execute single GET/POST commands to understand how the API is working.

GET request: View all Service BDCS-CE instances

/paas/api/v1.1/instancemgmt/{identityDomainId}/services/BDCSCE/instances

curl -i -X GET \
        -u "username:password" \
        -H "X-ID-TENANT-NAME: axxxxxx" \
        -H "Accept: application/json" \
        "https://psm.europe.oraclecloud.com/paas/api/v1.1/instancemgmt/axxxxxx/services/BDCSCE/instances"

Result:

HTTP/1.1 200 OK
Server: Oracle-Application-Server-11g
Strict-Transport-Security: max-age=31536000;includeSubDomains
Content-Language: en
...

{"services":{"cluster-iot":{"...

According to the HTTP status code, the command was successful.

GET request: View a specific Service BDCS-CE instances

Add the instance name to get the status of a specific cluster. Note that a BDCS-CE instance is your Big Data cluster.

/paas/api/v1.1/instancemgmt/{identityDomainId}/services/BDCSCE/instances/{instanceName}

curl -i -X GET \
        -u "username:password" \
        -H "X-ID-TENANT-NAME: axxxxxx" \
        -H "Accept: application/json" \
        "https://psm.europe.oraclecloud.com/paas/api/v1.1/instancemgmt/axxxxxx/services/BDCSCE/instances/cluster-iot"

Then use the same requests structure to start/stop/restart your Big Data cluster.

POST request: Start / Stop / Restart Service Instances BDCS-CE: cluster-iot

/paas/api/v1.1/instancemgmt/{identityDomainId}/services/BDCSCE/instances/{instanceName}/hosts/{function}

As it’s specified in the documentation, you need to change the media type to application/vnd.com.oracle.oracloud.provisioning.Service+json and use a body parameter to specify which hosts you want to manage. In our case, we want to manage all cluster hosts.

curl -i -X POST -u "username:password" \
-H "X-ID-TENANT-NAME: axxxxxx" \
-H "Content-Type: application/vnd.com.oracle.oracloud.provisioning.Service+json" \
-d '{"allServiceHosts":"true"}' "https://psm.europe.oraclecloud.com/paas/api/v1.1/instancemgmt/axxxxxx/services/BDCSCE/instances/cluster-iot/hosts/stop"

You can now, start to develop an automation script to manage your Oracle Big Data Compute Edition cluster.

Python prerequisites:

Install Python-PIP before:

dbi@host:~/$ sudo apt-get install python-pip

Install Requests module with PIP:

dbi@host:~/$ sudo pip install requests

Code:

#!/usr/bin/env python
# -*- coding: utf-8 -*-
"""
__author__ = "Mehdi Bada"
__company__= dbi services sa
__version__ = "1.0"
__maintainer__ = "Mehdi Bada"
__email__ = ""
__status__ = "Dev"

"""

import os, sys, getopt
import requests
import simplejson
import json

# Variables defintion.

identityDomainId="axxxxxx"
instanceName="cluster-iot"
server = "https://psm.europe.oraclecloud.com"
commands = ['start', 'stop', 'restart']


def usage():
    print "\nScript Usage \n"
    print "Usage:", sys.argv[0], "-c [start|stop|restart] | -h \n"

if len(sys.argv) < 3:
    usage()
    sys.exit(2)

try:
    opts, args = getopt.getopt(sys.argv[1:], "ch", ["command", "help"])
except getopt.GetoptError:
    usage()
    sys.exit(2)

for opt, arg in opts:
    if opt in ("-h", "--help"):
        usage()
        sys.exit()
    elif opt in ("-c", "--command"):
        icommand=sys.argv[2]
        if icommand in commands:
                icommand=sys.argv[2]
        else:
                usage()
                sys.exit(2)


url = server + "/paas/api/v1.1/instancemgmt/%s/services/BDCSCE/instances/%s/hosts/%s" % (identityDomainId,instanceName,icommand)

payload = "{\"allServiceHosts\":\"true\"}"

headers = {
    'x-id-tenant-name': "%s" %(identityDomainId),
    'accept': "application/vnd.com.oracle.oracloud.provisioning.Service+json",
    'content-type': "application/json",
    'authorization': " ",
    }

response = requests.request("POST", url, data=payload, headers=headers)

# Print the status code of the response.
print("\n")
print(response.status_code)

# Json Parsing
content=response.content
j = simplejson.loads(content)
print (j['details']['message'])

Usage:

dbi@host:~/$ ./bdcsce_start_stop_test.py -h

Script Usage

Usage: ./bdcsce_start_stop_test.py -c [start|stop|restart] | -h

 

Conclusion

Oracle REST API is not very well documented, that’s why multiple tests should be performed before understanding how it works.

 

3 Comments

  • Thomas Teske says:

    Dear Mehdi,
    thanks for sharing the first impression of the REST API. We are always keen on improving services and documentation thereof. Just PING me for the kind of improvements you have in mind. Is it about the wy we describe it, the lack of examples or something else?
    Kind regards
    Thomas

     
  • Thanks for your feedback. I run the Product Management group for Oracle’s cloud-native Big Data Services – BDCS-CE and EHCS. This is good feedback. We will enhance our documentation with more examples, tutorials etc to make it easier for folks such as yourself to get started. Please feel free to reach out directly to me – if you have any Product Related feedback.

     

Leave a Reply


7 × one =

Mehdi Bada
Mehdi Bada

Consultant