Showing posts with label Engineered Systems. Show all posts
Showing posts with label Engineered Systems. Show all posts

Tuesday, July 22, 2014

SOA domain creation on Exalogic using Exadata

I had afew lesson learn while doing Oracle SOA Suite 11.1.1.7 platform provisioning on Exalogic using Exadata. These configurations will maximise your performance on your Oracle Engineered Systems investment.

Lesson #1:

Repository Creation Utility (RCU) has a limitation when it comes to Sockets Direct Protocol (SDP). Paul Done wrote a good article on which version of JVM is supports SDP.

The root cause with RCU is that the SDP classes are not in the startup therefore RCU only works with TCP. Therefore it is a recommended best practice to run the RCU step using the TCP during installation, example connection string below.

jdbc:oracle:thin:@(DESCRIPTION_LIST=(DESCRIPTION=(CONNECT_TIMEOUT=3)(TRANSPORT_CONNECT_TIMEOUT=3)(ADDRESS_LIST=(LOAD_BALANCE=on)(ADDRESS=(PROTOCOL=TCP)(HOST=db-node1-ib)(PORT=1522))(ADDRESS=(PROTOCOL=TCP)(HOST=db-node2-ib)(PORT=1522)))(CONNECT_DATA=(SERVICE_NAME=SOASID))))

Once the domain has been created and running, goto each of the JDBC Data Sources and update all connection string to SDP.

jdbc:oracle:thin:@(DESCRIPTION_LIST=(DESCRIPTION=(CONNECT_TIMEOUT=3)(TRANSPORT_CONNECT_TIMEOUT=3)(ADDRESS_LIST=(LOAD_BALANCE=on)(ADDRESS=(PROTOCOL=SDP)(HOST=db-node1-ib)(PORT=1522))(ADDRESS=(PROTOCOL=SDP)(HOST=db-node2-ib)(PORT=1522)))(CONNECT_DATA=(SERVICE_NAME=SOASID))))

Lesson #2:

Modify setDomainEnv.sh to add support for SDP, this setting will be enabled for all servers in the current domain.

JAVA_OPTIONS="${JAVA_OPTIONS} -Doracle.net.SDP=true"

Lesson #3:

Managed Server is recommended to mounted exclusively by each compute node (ie. local volume). This is mention explicitly in the Exalogic Elastic Cloud Enterprise Deployment Guide for Oracle SOA Suite. This is to optimise I/O performance to disk by leveraging on Exalogic solid-state drive (SSD) technology.

However, the trade off is shared storage (NFS), shared storage enables reliability and availability through data replication and DR.

Lesson #4:

Ensures that Enable Exalogic Optimizations is selected under Domain > Configuration > General.

Refers to Oracle Support Note How To Enable Exalogic Optimizations For WebLogic Server Running On Exalogic Machine (Doc ID 1373571.1) for up-to-date details.

Database connectivity testing over SDP using Exadata

Optimising performance with Oracle Engineered Systems (Exadata). When running application such as SOA Suite, Siebel, JDE etc. connecting to database tier using Exadata.

Oracle best practice is that the Exadata is usually setup with a listener on port 1521 on the ETH / SCAN NIC(s) and on the IBoIP should be two listeners on port 1522 for TCP & SDP.

Below are steps to run on the application tier (soa-hostname in this case) to check the prerequisites before commencing product installation such as RCU.

Login to application server as oracle.
login as: oracle
oracle@soa-hostname's password:
Last login: Tue Jul 22 10:40:43 2014 from local-workstation
|-----------------------------------------------------------------|
| This system is for the use of authorized users only.            |
| Individuals using this computer system without authority, or in |
| excess of their authority, are subject to having all of their   |
| activities on this system monitored and recorded by system      |
| personnel.                                                      |
|                                                                 |
| In the course of monitoring individuals improperly using this   |
| system, or in the course of system maintenance, the activities  |
| of authorized users may also be monitored.                      |
|                                                                 |
| Anyone using this system expressly consents to such monitoring  |
| and is advised that if such monitoring reveals possible         |
| evidence of criminal activity, system personnel may provide the |
| evidence of such monitoring to law enforcement officials.       |
|-----------------------------------------------------------------|


Set the oracle environment variables to work as a client to databases.
-bash-3.2$ . oraenv
ORACLE_SID = [oracle] ? client
The Oracle base has been set to /opt/oracle/11.2.0.3


Change directory to the location of the file tnsnames.ora.
-bash-3.2$ cd $ORACLE_HOME/network/admin

Here is where you should be. This may varies based on where your DBA installed the client.
-bash-3.2$ pwd
/opt/oracle/11.2.0.3/client/network/admin


Check the file exists, if not it will have to be created. Ask your DBA for the tnsnames.ora file.
-bash-3.2$ ls
samples  shrept.lst  tnsnames.ora


These are the current settings for connection to the SOASID database via InfiniBand IPoIB (SDP) and E0IB (TCP).
-bash-3.2$ cat tnsnames.ora
SOASID_EXALOGIC =
        (DESCRIPTION_LIST=
                (LOAD_BALANCE=off)
                (FAILOVER=on)
                (FAILOVER_MODE = (TYPE = session)(METHOD = BASIC)(RETRIES = 2)(DELAY = 3))
                (DESCRIPTION=
                        (CONNECT_TIMEOUT=3)(TRANSPORT_CONNECT_TIMEOUT=3)
                        (ADDRESS_LIST= (LOAD_BALANCE=on) (ADDRESS=(PROTOCOL=SDP)(HOST=db-node1-ib)(PORT=1522)) (ADDRESS=(PROTOCOL=SDP)(HOST=db-node2-ib)(PORT=1522)) )
                        (CONNECT_DATA=(SERVICE_NAME= SOASID))
                )
        )

SOASID_CLIENT =
        (DESCRIPTION_LIST=
                (DESCRIPTION=
                        (ADDRESS_LIST= (ADDRESS=(PROTOCOL=TCP)(HOST=nnn.nnn.nnn.nnn)(PORT=1521)))
                        (CONNECT_DATA=(SERVICE_NAME= SOASID))
                )
        )


Check the listener can be contacted on IPoIB (SDP).  The numeric 8 at the end of the command is the number of pings.
-bash-3.2$ tnsping soasid_exalogic 8

TNS Ping Utility for Linux: Version 11.2.0.3.0 - Sandpit on 22-JUL-2014 11:25:01

Copyright (c) 1997, 2011, Oracle.  All rights reserved.

Used parameter files:


Used TNSNAMES adapter to resolve the alias
Attempting to contact (DESCRIPTION_LIST= (LOAD_BALANCE=off) (FAILOVER=on) (FAILOVER_MODE = (TYPE = session)(METHOD = BASIC)(RETRIES = 2)(DELAY = 3)) (DESCRIPTION= (CONNECT_TIMEOUT=3)(TRANSPORT_CONNECT_TIMEOUT=3) (ADDRESS_LIST= (LOAD_BALANCE=on) (ADDRESS=(PROTOCOL=SDP)(HOST=db-node1-ib)(PORT=1522)) (ADDRESS=(PROTOCOL=SDP)(HOST=db-node2-ib)(PORT=1522))) (CONNECT_DATA=(SERVICE_NAME= SOASID))))
OK (10 msec)
OK (10 msec)
OK (10 msec)
OK (10 msec)
OK (10 msec)
OK (20 msec)
OK (10 msec)
OK (10 msec)


Check the listener can be contacted on EoIB (TCP).
-bash-3.2$ tnsping soasid_client 8

TNS Ping Utility for Linux: Version 11.2.0.3.0 - Sandpit on 22-JUL-2014 11:25:10

Copyright (c) 1997, 2011, Oracle.  All rights reserved.

Used parameter files:


Used TNSNAMES adapter to resolve the alias
Attempting to contact (DESCRIPTION_LIST= (DESCRIPTION= (ADDRESS_LIST= (ADDRESS=(PROTOCOL=TCP)(HOST=nnn.nnn.nnn.nnn)(PORT=1521))) (CONNECT_DATA=(SERVICE_NAME= SOASID))))
OK (0 msec)
OK (0 msec)
OK (0 msec)
OK (0 msec)
OK (10 msec)
OK (0 msec)
OK (0 msec)
OK (0 msec)


Connect to the database service via IPoIB & verify the account has sysdba privileges.
-bash-3.2$ sqlplus sysdba/sysdba@soasid_exalogic as sysdba

SQL*Plus: Release 11.2.0.3.0 Sandpit on Tue Jul 22 11:25:52 2014

Copyright (c) 1982, 2011, Oracle.  All rights reserved.


Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Sandpit
With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP,
Data Mining and Real Application Testing options

SQL> exit
Disconnected from Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Sandpit
With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP,
Data Mining and Real Application Testing options


Connect to the database service via EoIB.
-bash-3.2$ sqlplus sysdba/sysdba@soasid_client as sysdba

SQL*Plus: Release 11.2.0.3.0 Sandpit on Tue Jul 22 11:26:07 2014

Copyright (c) 1982, 2011, Oracle.  All rights reserved.


Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Sandpit
With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP,
Data Mining and Real Application Testing options

SQL> exit
Disconnected from Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Sandpit
With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP,
Data Mining and Real Application Testing options


If you've reached this far it means connectivity test to your database via SDP on Exadata is successful!

Next step, run your Oracle Repository Creation Utility (RCU).

Here are some common issues:
  • DBA have not follow Oracle Support Note on Setup Listener on Infiniband Network using both SDP and TCP Protocol (Doc ID 1580584.1).
  • Some customer choose to run SDP and TCP on port 1521.
  • iptables on the VM may block the port(s) to the database.