Friday, December 6, 2013

Oracle Coherence

Oracle Coherence In-Memory Data Grid is a Data Management System for application objects that are shared across multiple Servers.

Data Grid – is a system which contains multiple servers that actually work together to manage information and operations related to them (computations) in a Distributed Environment.

In Memory Data Gird which stores information in memory which helps in achieving high performance. This keeps copies of the information in sync with all other servers ( in Cluster) to ensure data is available at all the time . Since the information is saved with all other server ,the data will be available even when a server is down.

Application Objects are the components that the application use to store information .these objects are shared across multiple servers and should survive server failures.

The In-Memory Data Grid achieve low response times for data access by keeping the information in memory which can be accessed by the application when it needed in an application object form. The information is stored with all servers in the cluster, this makes the application to access data from any of the server in cluster.

Servers can be added to the cluster for better performance, Since information is synchronously replicated across multiple servers, so no Single Point of Failure (SPOF) exists.

Each server knows where the synchronous replicas of each piece of information are located, and automatically re-routes information access and change operations to those replicas.

Oracle Coherence ensures that each operation executes in a Once-and-Only-Once manner, so that operations that are being executed when a server fails do not accidentally corrupt information during failover.

When our applications need to process large amounts of data and also need good response times , scalability , availability then Oracle coherence is a useful component.

In this article, we will see the basics of the Oracle Coherence configuration

1. Download the Coherence Package from here

2. Once we got the Coherence package, we can see 3 directories in there
Bin: contains the Scripts For starting and Stopping of Coherence
Lib: contains Libraries
Doc: Documentation.

3. The Coherence package contains 2 shell scripts which helps to identify whether coherence works fine or not.

The "cache server," is a simple application that hosts and manages data on behalf of other applications in a cluster.

The "coherence shell," is a simple application that enables a developer to access, process, and update cached data within a cluster. Information about the Cluster can also be obtained from here. By executing these applications on either a single host or several hosts, you can determine whether Coherence is operating correctly locally or across a network.

Starting the Coherence Cache Server
The cache-server can be started using the cache-server.sh script. Before starting the server , we need to change a few settings into to work.

Export COHERENCE_HOME with the path of the coherence home location and add the Coherence bin location to the path.

export COHERENCE_HOME=/soa/coherence
export PATH=$PATH://soa/coherence/bin

Once the variable are added ,Now lets change the sh files
Open the cache-server.sh and change COHERENCE_HOME,
COHERENCE_HOME=/soa/coherence/;

Now execute the cache-server.sh

[root@vx111a bin]# sh cache-server.sh
java version "1.7.0_25"
Java(TM) SE Runtime Environment (build 1.7.0_25-b15)
Java HotSpot(TM) 64-Bit Server VM (build 23.25-b01, mixed mode)

2013-12-06 16:54:09.340/0.340 Oracle Coherence 3.7.1.0 <Info> (thread=main, member=n/a): Loaded operational configuration from "jar:file:/soa/coherence/lib/coherence.jar!/tangosol-coherence.xml"
2013-12-06 16:54:09.407/0.407 Oracle Coherence 3.7.1.0 <Info> (thread=main, member=n/a): Loaded operational overrides from "jar:file:/soa/coherence/lib/coherence.jar!/tangosol-coherence-override-dev.xml"
2013-12-06 16:54:09.408/0.408 Oracle Coherence 3.7.1.0 <D5> (thread=main, member=n/a): Optional configuration override "/tangosol-coherence-override.xml" is not specified
2013-12-06 16:54:09.412/0.412 Oracle Coherence 3.7.1.0 <D5> (thread=main, member=n/a): Optional configuration override "/custom-mbeans.xml" is not specified

Oracle Coherence Version 3.7.1.0 Build 27797
 Grid Edition: Development mode
Copyright (c) 2000, 2011, Oracle and/or its affiliates. All rights reserved.

2013-12-06 16:54:09.678/0.678 Oracle Coherence GE 3.7.1.0 <Info> (thread=main, member=n/a): Loaded cache configuration from "jar:file:/soa/coherence/lib/coherence.jar!/coherence-cache-config.xml"
2013-12-06 16:54:10.163/1.163 Oracle Coherence GE 3.7.1.0 <Warning> (thread=main, member=n/a): UnicastUdpSocket failed to set receive buffer size to 16 packets (1023KB); actual size is 12%, 2 packets (127KB). Consult your OS documentation regarding increasing the maximum socket buffer size. Proceeding with the actual value may cause sub-optimal performance.
2013-12-06 16:54:10.163/1.163 Oracle Coherence GE 3.7.1.0 <Warning> (thread=main, member=n/a): PreferredUnicastUdpSocket failed to set receive buffer size to 1444 packets (1.99MB); actual size is 6%, 90 packets (127KB). Consult your OS documentation regarding increasing the maximum socket buffer size. Proceeding with the actual value may cause sub-optimal performance.
2013-12-06 16:54:10.163/1.163 Oracle Coherence GE 3.7.1.0 <D4> (thread=main, member=n/a): TCMP bound to /172.16.101.68:8088 using SystemSocketProvider
2013-12-06 16:54:10.164/1.164 Oracle Coherence GE 3.7.1.0 <Warning> (thread=main, member=n/a): MulticastUdpSocket failed to set receive buffer size to 16 packets (1023KB); actual size is 12%, 2 packets (127KB). Consult your OS documentation regarding increasing the maximum socket buffer size. Proceeding with the actual value may cause sub-optimal performance.
2013-12-06 16:54:13.429/4.429 Oracle Coherence GE 3.7.1.0 <Info> (thread=Cluster, member=n/a): Created a new cluster "cluster:0xFCDB" with Member(Id=1, Timestamp=2013-12-06 16:54:10.201, Address=172.16.101.68:8088, MachineId=55620, Location=site:,process:4722, Role=CoherenceServer, Edition=Grid Edition, Mode=Development, CpuCount=4, SocketCount=2) UID=0xAC10654400000142C7A68C59D9441F98
2013-12-06 16:54:13.434/4.434 Oracle Coherence GE 3.7.1.0 <Info> (thread=main, member=n/a): Started cluster Name=cluster:0xFCDB

Group{Address=224.3.7.0, Port=37000, TTL=4}

MasterMemberSet(
  ThisMember=Member(Id=1, Timestamp=2013-12-06 16:54:10.201, Address=172.16.101.68:8088, MachineId=55620, Location=site:,process:4722, Role=CoherenceServer)
  OldestMember=Member(Id=1, Timestamp=2013-12-06 16:54:10.201, Address=172.16.101.68:8088, MachineId=55620, Location=site:,process:4722, Role=CoherenceServer)
  ActualMemberSet=MemberSet(Size=1
    Member(Id=1, Timestamp=2013-12-06 16:54:10.201, Address=172.16.101.68:8088, MachineId=55620, Location=site:,process:4722, Role=CoherenceServer)
    )
  MemberId|ServiceVersion|ServiceJoined|MemberState
    1|3.7.1|2013-12-06 16:54:13.429|JOINED
  RecycleMillis=1200000
  RecycleSet=MemberSet(Size=0
    )
  )

TcpRing{Connections=[]}
IpMonitor{AddressListSize=0}

2013-12-06 16:54:13.472/4.472 Oracle Coherence GE 3.7.1.0 <D5> (thread=Invocation:Management, member=1): Service Management joined the cluster with senior service member 1
2013-12-06 16:54:13.627/4.627 Oracle Coherence GE 3.7.1.0 <D5> (thread=DistributedCache, member=1): Service DistributedCache joined the cluster with senior service member 1
2013-12-06 16:54:13.701/4.701 Oracle Coherence GE 3.7.1.0 <D5> (thread=ReplicatedCache, member=1): Service ReplicatedCache joined the cluster with senior service member 1
2013-12-06 16:54:13.710/4.710 Oracle Coherence GE 3.7.1.0 <D5> (thread=OptimisticCache, member=1): Service OptimisticCache joined the cluster with senior service member 1
2013-12-06 16:54:13.716/4.716 Oracle Coherence GE 3.7.1.0 <D5> (thread=Invocation:InvocationService, member=1): Service InvocationService joined the cluster with senior service member 1
2013-12-06 16:54:13.719/4.719 Oracle Coherence GE 3.7.1.0 <Info> (thread=main, member=1):
Services
  (
  ClusterService{Name=Cluster, State=(SERVICE_STARTED, STATE_JOINED), Id=0, Version=3.7.1, OldestMemberId=1}
  InvocationService{Name=Management, State=(SERVICE_STARTED), Id=1, Version=3.1, OldestMemberId=1}
  PartitionedCache{Name=DistributedCache, State=(SERVICE_STARTED), LocalStorage=enabled, PartitionCount=257, BackupCount=1, AssignedPartitions=257, BackupPartitions=0}
  ReplicatedCache{Name=ReplicatedCache, State=(SERVICE_STARTED), Id=3, Version=3.0, OldestMemberId=1}
  Optimistic{Name=OptimisticCache, State=(SERVICE_STARTED), Id=4, Version=3.0, OldestMemberId=1}
  InvocationService{Name=InvocationService, State=(SERVICE_STARTED), Id=5, Version=3.1, OldestMemberId=1}
  )

Started DefaultCacheServer...

Once we see the Started DefaultCacheServer, we are sure that the cache-server is started.

Starting the Coherence
Once the cache-server is started, now let start the Coherence shell.For this to start we need to make a few Changes,

# specify if the console will also act as a server
STORAGE_ENABLED=true

Make sure that the STORAGE_ENABLED is set to true. If this is not set to true we see messages like,
com.tangosol.net.RequestPolicyException: No storage-enabled nodes exist for service DistributedCache

And we can’t put data into the cache.

Coherence requires at least one storage enabled server in the cluster. The cache server you started is not storage enabled. As an example, in .\bin directory of the coherence install, there is a coherence.sh. By default, it is not storage enabled. You can run cache-server.cmd to
start a storage enabled cache server.. Alternatively, you can edit coherence.cmd to change "set storage_enabled=false" to "set storage_enabled=true". Then you should be able to put data into the cache from the coherence.cmd command prompt.

Once the Changes are done, start the shell using
[root@vx111a bin]# sh coherence.sh
** Starting storage enabled console **
java version "1.7.0_25"
Java(TM) SE Runtime Environment (build 1.7.0_25-b15)
Java HotSpot(TM) 64-Bit Server VM (build 23.25-b01, mixed mode)

2013-12-06 17:14:52.201/0.395 Oracle Coherence 3.7.1.0 <Info> (thread=main, member=n/a): Loaded operational configuration from "jar:file:/soa/coherence/lib/coherence.jar!/tangosol-coherence.xml"
2013-12-06 17:14:52.292/0.486 Oracle Coherence 3.7.1.0 <Info> (thread=main, member=n/a): Loaded operational overrides from "jar:file:/soa/coherence/lib/coherence.jar!/tangosol-coherence-override-dev.xml"
2013-12-06 17:14:52.292/0.486 Oracle Coherence 3.7.1.0 <D5> (thread=main, member=n/a): Optional configuration override "/tangosol-coherence-override.xml" is not specified
2013-12-06 17:14:52.298/0.492 Oracle Coherence 3.7.1.0 <D5> (thread=main, member=n/a): Optional configuration override "/custom-mbeans.xml" is not specified

Oracle Coherence Version 3.7.1.0 Build 27797
Grid Edition: Development mode
Copyright (c) 2000, 2011, Oracle and/or its affiliates. All rights reserved.

2013-12-06 17:14:53.063/1.257 Oracle Coherence GE 3.7.1.0 <Warning> (thread=main, member=n/a): UnicastUdpSocket failed to set receive buffer size to 16 packets (1023KB); actual size is 12%, 2 packets (127KB). Consult your OS documentation regarding increasing the maximum socket buffer size. Proceeding with the actual value may cause sub-optimal performance.
2013-12-06 17:14:53.063/1.257 Oracle Coherence GE 3.7.1.0 <Warning> (thread=main, member=n/a): PreferredUnicastUdpSocket failed to set receive buffer size to 1444 packets (1.99MB); actual size is 6%, 90 packets (127KB). Consult your OS documentation regarding increasing the maximum socket buffer size. Proceeding with the actual value may cause sub-optimal performance.
2013-12-06 17:14:53.063/1.257 Oracle Coherence GE 3.7.1.0 <D4> (thread=main, member=n/a): TCMP bound to /172.16.101.68:8090 using SystemSocketProvider
2013-12-06 17:14:53.063/1.257 Oracle Coherence GE 3.7.1.0 <Warning> (thread=main, member=n/a): MulticastUdpSocket failed to set receive buffer size to 16 packets (1023KB); actual size is 12%, 2 packets (127KB). Consult your OS documentation regarding increasing the maximum socket buffer size. Proceeding with the actual value may cause sub-optimal performance.
2013-12-06 17:14:56.329/4.523 Oracle Coherence GE 3.7.1.0 <Info> (thread=Cluster, member=n/a): Created a new cluster "cluster:0xFCDB" with Member(Id=1, Timestamp=2013-12-06 17:14:53.1, Address=172.16.101.68:8090, MachineId=55620, Location=site:,process:5124, Role=CoherenceConsole, Edition=Grid Edition, Mode=Development, CpuCount=4, SocketCount=2) UID=0xAC10654400000142C7B9836CD9441F9A
2013-12-06 17:14:56.333/4.527 Oracle Coherence GE 3.7.1.0 <Info> (thread=main, member=n/a): Started cluster Name=cluster:0xFCDB

Group{Address=224.3.7.0, Port=37000, TTL=4}

MasterMemberSet(
  ThisMember=Member(Id=1, Timestamp=2013-12-06 17:14:53.1, Address=172.16.101.68:8090, MachineId=55620, Location=site:,process:5124, Role=CoherenceConsole)
  OldestMember=Member(Id=1, Timestamp=2013-12-06 17:14:53.1, Address=172.16.101.68:8090, MachineId=55620, Location=site:,process:5124, Role=CoherenceConsole)
  ActualMemberSet=MemberSet(Size=1
    Member(Id=1, Timestamp=2013-12-06 17:14:53.1, Address=172.16.101.68:8090, MachineId=55620, Location=site:,process:5124, Role=CoherenceConsole)
    )
  MemberId|ServiceVersion|ServiceJoined|MemberState
    1|3.7.1|2013-12-06 17:14:56.329|JOINED
  RecycleMillis=1200000
  RecycleSet=MemberSet(Size=0
    )
  )

TcpRing{Connections=[]}
IpMonitor{AddressListSize=0}

2013-12-06 17:14:56.367/4.561 Oracle Coherence GE 3.7.1.0 <D5> (thread=Invocation:Management, member=1): Service Management joined the cluster with senior service member 1

Map(?):

Once you see the prompt mode , we can add cache and data to it like,

Map (?): cache test
2013-12-06 17:15:00.968/9.162 Oracle Coherence GE 3.7.1.0 <Info> (thread=main, member=1): Loaded cache configuration from "jar:file:/soa/coherence/lib/coherence.jar!/coherence-cache-config.xml"
2013-12-06 17:15:01.181/9.375 Oracle Coherence GE 3.7.1.0 <D5> (thread=DistributedCache, member=1): Service DistributedCache joined the cluster with senior service member 1
<distributed-scheme>
  <scheme-name>example-distributed</scheme-name>
  <service-name>DistributedCache</service-name>
  <backing-map-scheme>
    <local-scheme>
      <scheme-ref>example-binary-backing-map</scheme-ref>
    </local-scheme>
  </backing-map-scheme>
  <autostart>true</autostart>
</distributed-scheme>

Map (test): put foo bar
null

Map (test): get foo
Bar

Or

Map (?): maps animal

Map (?): cache animal
2013-12-06 19:27:36.675/13.308 Oracle Coherence GE 3.7.1.0 <Info> (thread=main, member=1): Loaded cache configuration from "jar:file:/soa/coherence/lib/coherence.jar!/coherence-cache-config.xml"
2013-12-06 19:27:36.895/13.528 Oracle Coherence GE 3.7.1.0 <D5> (thread=DistributedCache, member=1): Service DistributedCache joined the cluster with senior service member 1
<distributed-scheme>
  <scheme-name>example-distributed</scheme-name>
  <service-name>DistributedCache</service-name>
  <backing-map-scheme>
    <local-scheme>
      <scheme-ref>example-binary-backing-map</scheme-ref>
    </local-scheme>
  </backing-map-scheme>
  <autostart>true</autostart>
</distributed-scheme>

Map (animal): put 1 cat
null

Map (animal): put 2 dog
null

Map (animal): list
1 = cat
2 = dog

Map (animal): get 1
cat

Map (animal): get 2
dog

Map (animal): put 3 bird
null

Map (animal): list
1 = cat
2 = dog
3 = bird

Coherence Query Language
CohQL is a new light-weight syntax (in the tradition of SQL) that is used to perform cache operations on a Coherence cluster. The language can be used either programmatically or from a command-line tool.

Coherence package provides a shell script to invoke the COhQL mode.This can be invoked using query.sh script. Before starting the script we need to make some changes.

Change,
# specify if the console will also act as a server
STORAGE_ENABLED=true

COHERENCE_HOME=/soa/coherence/;

The JLINE_HOME environment variable should be set to the location of the JLine JAR, which is used for enhanced command-line editing capabilities, such as having the up and down arrows move through the command history. Use the Default value provided.

[root@vx111a bin]# sh query.sh
** Starting storage enabled console **
java version "1.7.0_25"
Java(TM) SE Runtime Environment (build 1.7.0_25-b15)
Java HotSpot(TM) 64-Bit Server VM (build 23.25-b01, mixed mode)

Coherence Command Line Tool

CohQL> create cache "products"
2013-12-06 17:05:29.405/18.930 Oracle Coherence 3.7.1.0 <Info> (thread=main, member=n/a): Loaded operational configuration from "jar:file:/soa/coherence/lib/coherence.jar!/tangosol-coherence.xml"
2013-12-06 17:05:29.507/19.032 Oracle Coherence 3.7.1.0 <Info> (thread=main, member=n/a): Loaded operational overrides from "jar:file:/soa/coherence/lib/coherence.jar!/tangosol-coherence-override-dev.xml"
2013-12-06 17:05:29.508/19.033 Oracle Coherence 3.7.1.0 <D5> (thread=main, member=n/a): Optional configuration override "/tangosol-coherence-override.xml" is not specified
2013-12-06 17:05:29.513/19.038 Oracle Coherence 3.7.1.0 <D5> (thread=main, member=n/a): Optional configuration override "/custom-mbeans.xml" is not specified

Oracle Coherence Version 3.7.1.0 Build 27797
 Grid Edition: Development mode
Copyright (c) 2000, 2011, Oracle and/or its affiliates. All rights reserved.

2013-12-06 17:05:29.801/19.326 Oracle Coherence GE 3.7.1.0 <Info> (thread=main, member=n/a): Loaded cache configuration from "jar:file:/soa/coherence/lib/coherence.jar!/coherence-cache-config.xml"
2013-12-06 17:05:30.317/19.842 Oracle Coherence GE 3.7.1.0 <Warning> (thread=main, member=n/a): UnicastUdpSocket failed to set receive buffer size to 16 packets (1023KB); actual size is 12%, 2 packets (127KB). Consult your OS documentation regarding increasing the maximum socket buffer size. Proceeding with the actual value may cause sub-optimal performance.
2013-12-06 17:05:30.318/19.843 Oracle Coherence GE 3.7.1.0 <Warning> (thread=main, member=n/a): PreferredUnicastUdpSocket failed to set receive buffer size to 1444 packets (1.99MB); actual size is 6%, 90 packets (127KB). Consult your OS documentation regarding increasing the maximum socket buffer size. Proceeding with the actual value may cause sub-optimal performance.
2013-12-06 17:05:30.318/19.843 Oracle Coherence GE 3.7.1.0 <D4> (thread=main, member=n/a): TCMP bound to /172.16.101.68:8090 using SystemSocketProvider
2013-12-06 17:05:30.318/19.843 Oracle Coherence GE 3.7.1.0 <Warning> (thread=main, member=n/a): MulticastUdpSocket failed to set receive buffer size to 16 packets (1023KB); actual size is 12%, 2 packets (127KB). Consult your OS documentation regarding increasing the maximum socket buffer size. Proceeding with the actual value may cause sub-optimal performance.
2013-12-06 17:05:33.586/23.111 Oracle Coherence GE 3.7.1.0 <Info> (thread=Cluster, member=n/a): Created a new cluster "Cluster" with Member(Id=1, Timestamp=2013-12-06 17:05:30.355, Address=172.16.101.68:8090, MachineId=55620, Location=site:,process:4984, Role=TangosolCoherenceQueryPlus, Edition=Grid Edition, Mode=Development, CpuCount=4, SocketCount=2) UID=0xAC10654400000142C7B0ED33D9441F9A
2013-12-06 17:05:33.591/23.116 Oracle Coherence GE 3.7.1.0 <Info> (thread=main, member=n/a): Started cluster Name=Cluster

Group{Address=224.3.7.0, Port=37000, TTL=4}

MasterMemberSet(
  ThisMember=Member(Id=1, Timestamp=2013-12-06 17:05:30.355, Address=172.16.101.68:8090, MachineId=55620, Location=site:,process:4984, Role=TangosolCoherenceQueryPlus)
  OldestMember=Member(Id=1, Timestamp=2013-12-06 17:05:30.355, Address=172.16.101.68:8090, MachineId=55620, Location=site:,process:4984, Role=TangosolCoherenceQueryPlus)
  ActualMemberSet=MemberSet(Size=1
    Member(Id=1, Timestamp=2013-12-06 17:05:30.355, Address=172.16.101.68:8090, MachineId=55620, Location=site:,process:4984, Role=TangosolCoherenceQueryPlus)
    )
  MemberId|ServiceVersion|ServiceJoined|MemberState
    1|3.7.1|2013-12-06 17:05:33.586|JOINED
  RecycleMillis=1200000
  RecycleSet=MemberSet(Size=0
    )
  )

TcpRing{Connections=[]}
IpMonitor{AddressListSize=0}

2013-12-06 17:05:33.621/23.146 Oracle Coherence GE 3.7.1.0 <D5> (thread=Invocation:Management, member=1): Service Management joined the cluster with senior service member 1
2013-12-06 17:05:33.787/23.312 Oracle Coherence GE 3.7.1.0 <D5> (thread=DistributedCache, member=1): Service DistributedCache joined the cluster with senior service member 1

CohQL> create cache "products"

CohQL> insert into "products" key "telivision" value "TD-500"

CohQL> select * from "products"
Results
TD-500

CohQL> update "products" set value()="ID-501" where key() like "telivision"
Results
telivision: true

CohQL> select * from "products"
Results
ID-501

CohQL>

CohQL> select value(), key() from "products"
Results
"ID-501", "telivision"

That’s it about the basics of Oracle Coherence. In the next series we will see how to configure Oracle Coherence with Oracle Weblogic.

More to Come, Happy Coding.