[COH] 코히런스 데이터그램 테스트

이번 글에서는 Coherence datagram-test 하는 방법에 대해 소개하고자 한다.

How To Verify That Peer-To-Peer UDP Communication Does Work Well Between Machines Coherence Cluster Members Are Supposed To Be Running On (Doc ID 1936575.1)

> 이 문서는 Coherence와 함께 제공되는 데이터그램 테스트 유틸리티를 사용하는 방법을 예제로 설명한다.

 

* 기본적으로 Coherence 클러스터 구성원과 클러스터 구성원 간에 데이터를 보내기 위해 UDP 유니캐스트를 사용한다. (P2P)

 

1. 데이터그램 테스트 예제

================================================

1-1. 컴퓨팅 노드(물리적 서버) node1, node2, node3, node4가 있는 경우,

================================================

각 컴퓨팅 노드에는 서로 다른 서브넷에 바인딩된 두 개의 NIC가 있다.(Private: xx.xx.10.0/24, Public: xx.xx.1.0/24)

 

==============================================================

1-2. NIC가 각 컴퓨팅 노드에 있는 서브넷 xx.xx.10.0/24의 예상 IP주소에 바인딩되어 있는지 확인

==============================================================

[test@node1 coherence]$ ip addr list 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN     link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00     inet 127.0.0.1/8 scope host lo     inet6 ::1/128 scope host        valid_lft forever preferred_lft forever 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000     link/ether 08:00:27:08:be:93 brd ff:ff:ff:ff:ff:ff     inet 10.0.2.15/24 brd 10.0.2.255 scope global eth0     inet6 fe80::a00:27ff:fe08:be93/64 scope link        valid_lft forever preferred_lft forever 3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000     link/ether 08:00:27:37:0e:f9 brd ff:ff:ff:ff:ff:ff     inet xx.xx.1.11/24 brd xx.xx.1.255 scope global eth1     inet6 fe80::a00:27ff:fe37:ef9/64 scope link        valid_lft forever preferred_lft forever 4: eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000     link/ether 08:00:27:f9:18:9f brd ff:ff:ff:ff:ff:ff     inet xx.xx.10.11/24 brd xx.xx.10.255 scope global eth2     inet6 fe80::a00:27ff:fef9:189f/64 scope link        valid_lft forever preferred_lft forever [test@node2 coherence]$ ip addr list 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN     link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00     inet 127.0.0.1/8 scope host lo     inet6 ::1/128 scope host        valid_lft forever preferred_lft forever 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000     link/ether 08:00:27:27:d2:ce brd ff:ff:ff:ff:ff:ff     inet 10.0.2.15/24 brd 10.0.2.255 scope global eth0     inet6 fe80::a00:27ff:fe27:d2ce/64 scope link        valid_lft forever preferred_lft forever 3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000     link/ether 08:00:27:b3:e8:64 brd ff:ff:ff:ff:ff:ff     inet xx.xx.1.12/24 brd xx.xx.1.255 scope global eth1     inet6 fe80::a00:27ff:feb3:e864/64 scope link        valid_lft forever preferred_lft forever 4: eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000     link/ether 08:00:27:91:04:36 brd ff:ff:ff:ff:ff:ff     inet xx.xx.10.12/24 brd xx.xx.10.255 scope global eth2     inet6 fe80::a00:27ff:fe91:436/64 scope link        valid_lft forever preferred_lft forever ....

node1 : xx.xx.10.11/24

node2 : xx.xx.10.12/24

node3 : xx.xx.10.13/24

node4 : xx.xx.10.14/24

 

===================================================================

1-3. Coherence 클러스터 구성원이 실행되어야 하는 모든 컴퓨팅 노드 간의 UDP/데이터그램 통신 테스트

===================================================================

테스트 유틸리티는 모든 관련 컴퓨팅 노드(node1, node2, node3, node4)에서 실행되어야 한다.

단, node4에서 마지막으로 실행된 명령을 제외한 모든 명령은 -polite 옵션이 적용된다. 

(publisher/listener 둘 다 가능하게 하는 옵션)

 

node1:

bin/datagram-test.sh -polite -local xx.xx.10.11:8088 xx.xx.10.12:8088 xx.xx.10.13:8088 xx.xx.10.14:8088

 

node2:

bin/datagram-test.sh -polite -local xx.xx.10.12:8088 xx.xx.10.11:8088 xx.xx.10.13:8088 xx.xx.10.14:8088

 

node3:

bin/datagram-test.sh -polite -local xx.xx.10.13:8088 xx.xx.10.11:8088 xx.xx.10.12:8088 xx.xx.10.14:8088

 

node4:

bin/datagram-test.sh -local xx.xx.10.14:8088 xx.xx.10.11:8088 xx.xx.10.12:8088 xx.xx.10.13:8088

 

########################

1-4. 테스트 결과

########################

============================================================================== [test@node1 coherence]$ bin/datagram-test.sh -polite -local xx.xx.10.11:8088 xx.xx.10.12:8088 xx.xx.10.13:8088 xx.xx.10.14 java version "1.7.0_65" Java(TM) SE Runtime Environment (build 1.7.0_65-b17) Java HotSpot(TM) 64-Bit Server VM (build 24.65-b04, mixed mode) 2014-10-17 21:56:43.057/0.617 Oracle Coherence 3.7.1.13 <Info> (thread=main, member=n/a): Loaded operational configuration from "jar:file:[path]/coh-3.7.1.13/coherence/lib/coherence.jar!/tangosol-coherence.xml" 2014-10-17 21:56:43.196/0.756 Oracle Coherence 3.7.1.13 <Info> (thread=main, member=n/a): Loaded operational overrides from "jar:file:[path]/coh-3.7.1.13/coherence/lib/coherence.jar!/tangosol-coherence-override-dev.xml" 2014-10-17 21:56:43.202/0.762 Oracle Coherence 3.7.1.13 <D5> (thread=main, member=n/a): Optional configuration override "/tangosol-coherence-override.xml" is not specified 2014-10-17 21:56:43.206/0.766 Oracle Coherence 3.7.1.13 <D5> (thread=main, member=n/a): Optional configuration override "/custom-mbeans.xml" is not specified Oracle Coherence Version 3.7.1.13 Build 53621  Grid Edition: Development mode Copyright (c) 2000, 2014, Oracle and/or its affiliates. All rights reserved. creating datagram socket using provider: SystemSocketProvider starting listener: at /xx.xx.10.11:8088 packet size: 1468 bytes buffer size: 1428 packets   report on: 100000 packets, 139 MBs     process: 20 bytes/packet         log: null      log on: 139 MBs starting publisher: at /xx.xx.10.11:8089 sending to /xx.xx.10.12:8088, /xx.xx.10.13:8088, /xx.xx.10.14:9999 packet size: 1468 bytes buffer size: 32 packets   report on: 100000 packets, 139 MBs     process: 20 bytes/packet       peers: 3        rate: no limit no packet burst limit waiting for listener to be contacted before publishing Recieving data from 1 publisher(s). Recieving data from 2 publisher(s). Recieving data from 3 publisher(s). ioioioioiooioioiioIiOioioioiioioioioIoioiOioioioioioioioIoioiOiioioiooioioioIoioiOioioioioioioioIoioiiOoioioioioioioIoioiOioioiooioiioioIoioiOioioioioioioioIoioiOioioioioioioioIoioiOioioioioioioioI Lifetime: Rx from publisher: /xx.xx.10.12:8089              elapsed: 6862ms          packet size: 1468           throughput: 6 MB/sec                       4470 packets/sec             received: 30675 of 30675              missing: 0         success rate: 1.0         out of order: 0           avg offset: 0                 gaps: 0         avg gap size: 0         avg gap time: 0ms         avg ack time: 0.44599ms; acks 25158 Rx from publisher: /xx.xx.10.13:8089              elapsed: 6863ms          packet size: 1468           throughput: 6 MB/sec                       4356 packets/sec             received: 29898 of 29898              missing: 0         success rate: 1.0         out of order: 0           avg offset: 0                 gaps: 0         avg gap size: 0         avg gap time: 0ms         avg ack time: 0.447785ms; acks 24994 Rx from publisher: /xx.xx.10.14:8089              elapsed: 6873ms          packet size: 1468           throughput: 8 MB/sec                       5737 packets/sec             received: 39427 of 39427              missing: 0         success rate: 1.0         out of order: 0           avg offset: 0                 gaps: 0         avg gap size: 0         avg gap time: 0ms         avg ack time: -1.0E-6ms; acks 0 Rx Summary from 3 publisher(s):              elapsed: 6873ms           throughput: 20 MB/sec                       14550 packets/sec             received: 100000 of 100000              missing: 0         success rate: 1.0         out of order: 0           avg offset: 0                 gaps: 0         avg gap size: 0         avg gap time: 0ms         avg ack time: 0.446885ms; acks 50152 ==============================================================================

 

 

* 만약 장비 4대 중에 node4 방화벽이 올라가 있다면 아래와 같이 OutPut 메세지만 출력된다.

 

 

============================================================================== [test@node4 coherence]$ bin/datagram-test.sh -local xx.xx.10.14:8088 xx.xx.10.11:8088 xx.xx.10.12:8088 xx.xx.10.13:8088 java version "1.7.0_65" Java(TM) SE Runtime Environment (build 1.7.0_65-b17) Java HotSpot(TM) 64-Bit Server VM (build 24.65-b04, mixed mode) 2014-10-17 21:57:01.277/0.641 Oracle Coherence 3.7.1.13 <Info> (thread=main, member=n/a): Loaded operational configuration from "jar:file:[path]/coh-3.7.1.13/coherence/lib/coherence.jar!/tangosol-coherence.xml" 2014-10-17 21:57:01.413/0.777 Oracle Coherence 3.7.1.13 <Info> (thread=main, member=n/a): Loaded operational overrides from "jar:file:[path]/coh-3.7.1.13/coherence/lib/coherence.jar!/tangosol-coherence-override-dev.xml" 2014-10-17 21:57:01.420/0.784 Oracle Coherence 3.7.1.13 <D5> (thread=main, member=n/a): Optional configuration override "/tangosol-coherence-override.xml" is not specified 2014-10-17 21:57:01.425/0.789 Oracle Coherence 3.7.1.13 <D5> (thread=main, member=n/a): Optional configuration override "/custom-mbeans.xml" is not specified Oracle Coherence Version 3.7.1.13 Build 53621  Grid Edition: Development mode Copyright (c) 2000, 2014, Oracle and/or its affiliates. All rights reserved. creating datagram socket using provider: SystemSocketProvider starting listener: at /xx.xx.10.14:8088 packet size: 1468 bytes buffer size: 1428 packets   report on: 100000 packets, 139 MBs     process: 20 bytes/packet         log: null      log on: 139 MBs starting publisher: at /xx.xx.10.14:8089 sending to /xx.xx.10.11:8088, /xx.xx.10.12:8088, /xx.xx.10.13:8088 packet size: 1468 bytes buffer size: 32 packets   report on: 100000 packets, 139 MBs     process: 20 bytes/packet       peers: 3        rate: no limit no packet burst limit oooooooooOoooooooooOoooooooooOoooooooooOoooooooooOoooooooooOoooooooooOoooooooooOoooooooooOoooooooooO Tx summary 3 peers:    life: 24 MB/sec, 17053 packets/sec     now: 24 MB/sec, 17053 packets/sec oooooooooOoooooooooOoooooooooOoooooooooOoooooooooOoooooooooOoooooooooOoooooooooOoooooooooOoooooooooO Tx summary 3 peers:    life: 24 MB/sec, 17481 packets/sec     now: 25 MB/sec, 17931 packets/sec oooooooooOoooooooooOoooooooooOoooooooooOoooooooooOoooooooooOoooooooooOoooooooooOoooooooooOoooooooooO Tx summary 3 peers:    life: 24 MB/sec, 17486 packets/sec     now: 24 MB/sec, 17495 packets/sec oooooooooOoooooooooOoooooooooOoooooooooOoooooooooOoooooooooOoooooooooOoooooooooOoooooooooOoooooooooO Tx summary 3 peers:    life: 25 MB/sec, 17576 packets/sec     now: 25 MB/sec, 17854 packets/sec ==============================================================================

 

2. 데이터그램 테스트 통계

데이터 그램 테스트는 udp 통신이 가능한지 뿐만 아니라 네트워크 성능에 대한 통계를 확인할 수 있다.

 

# 샘플 통계 예제

=========================================== Lifetime: Rx from publisher: /195.0.0.2:9999              elapsed: 8770ms          packet size: 1468           throughput: 96 MB/sec                       68415 packets/sec             received: 600000 of 611400              missing: 11400         success rate: 0.9813543         out of order: 2           avg offset: 1 Now: Rx from publisher: /195.0.0.2:9999              elapsed: 1431ms          packet size: 1468           throughput: 98 MB/sec                       69881 packets/sec             received: 100000 of 100000              missing: 0         success rate: 1.0         out of order: 0           avg offset: 0 ===========================================

성능 통계에서 확인해야할 항목은 처리량과 성공률이다. 성공률은 가능한 1.0에 가깝게 유지하면서 가장 높은 처리량을 찾는 거이다.

보통 100Mb 네트워크 설정에서 약 10MB/초의 속도를 달성해야 하며, 1Gb 네트워크에서 약 100MB/초의 속도를 달성할 수 있어야 한다. 코히런스 POC나 BMT시에 성능 평가 및 테스트 용도로 확인하기 좋은 지표일 듯 싶다.