Infrastructure at your Service

Virtual Local Area Network (VLAN) have become since several years a standard in enterprise class networks. Most enterprises are now segregating their network, especially for security reasons, between server and user or prod and test or applications and backup and aso. In the new release of ODA we finally get the support for VLAN on the bare metal platform. This article will briefly demonstrate how these are managed using ODAADMCLI.

First of all we have to remember that VLAN is not brand new on ODA. While using ODA HA (X5-2 or X6-2) in virtual mode, which means with OVM, it was already possible to manage VLANs. However this was a bit different than the new feature introduced in ODA 12.2.1.2.0.

First of all the HA platform in virtual mode is running using OAKCLI and not ODACLI with the DCS agent. In background the real difference is that the HA in virtualized mode is using the Linux Bridge Control (more details here). To make it simple, you have a kind of virtual switches (bridges) on which each connected VM, including the ODA BASE, can be connected and get an address on this particular network.

On the bare metal platform the principle is totally different as it is directly based on the VLAN implementation in Linux (802.1q standard) which allows activating VLAN on an interface and tagging the packets with the right VLANID.

The first place where VLAN can be configured on the ODA is during the first network plumbing phase, right after booting or re-imaging the ODA, using the configure-firstnet command.

30-ILOM-Configure-FirstNet-VLAN

As shown above the command will ask you if you want to use VLAN or not. Answering YES will then request you to provide the VLANID for the primary network of the ODA and will generate a network interface btbond1.<VLANID>.

How does it looks like in background??

The first layer as always on ODA is a bonding of 2 physical interfaces (here em2 and em3 as I’m using the copper interfaces):

[[email protected] network-scripts]# cat ifcfg-em2
#File created by Oracle development
DEVICE=em2
ONBOOT=yes
BOOTPROTO=none
USERCTL=no
TYPE=ETHERNET
ETHTOOL_OFFLOAD_OPTS="lro off"
IPV6INIT=no
NM_CONTROLLED=no
PEERDNS=no
MASTER=btbond1
SLAVE=yes

Looking to the btbond1 interface we will see that it is configure in active-backup mode BUT without any IP address.

[[email protected] network-scripts]# cat ifcfg-btbond1
#File created by Oracle development
DEVICE=btbond1
ONBOOT=yes
BOOTPROTO=none
USERCTL=no
TYPE=BOND
BONDING_OPTS="mode=active-backup miimon=100 primary=em2"
IPV6INIT=no
NM_CONTROLLED=no
PEERDNS=no

On top of the bonding configuration, we have then a virtual interface per VLAN. Indeed only one at the beginning as the configure-firstnet generates only the “primary” network of the ODA.

[[email protected] network-scripts]# cat ifcfg-btbond1.54
#ODA_VLAN_CONFIG ===
#ODA_VLAN_CONFIG Name=vlan54
#ODA_VLAN_CONFIG VlanId=54
#ODA_VLAN_CONFIG VlanInterface=btbond1
#ODA_VLAN_CONFIG Type=VlanType
#ODA_VLAN_CONFIG VlanSetupType=public
#ODA_VLAN_CONFIG VlanIpAddr=192.168.54.10
#ODA_VLAN_CONFIG VlanNetmask=255.255.255.0
#ODA_VLAN_CONFIG VlanGateway=192.168.54.1
#ODA_VLAN_CONFIG NodeNum=0
#=== DO NOT EDIT ANYTHING ABOVE THIS LINE ===
DEVICE=btbond1.54
BOOTPROTO=none
ONBOOT=yes
VLAN=yes
NM_CONTROLLED=no
IPADDR=192.168.54.10
NETMASK=255.255.255.0
GATEWAY=192.168.54.1

Do not look for the VLANID in the configuration file (of, except in the comments  😉 ). It is defined by the device/file name.

Once you have your first VLAN you can easily configure additional ones using the command line. Remember that on the DCS stack ODAs you have 2 different CLIs: ODACLI and ODAADMCLI. The VLAN management is done using ODAADMCLI.

So lets have a look to the help:

[[email protected] ~]
# odaadmcli -h
Usage: odaadmcli <command> <object> [<options>]
 commands: show|manage|stordiag|power|expand
 objects : disk|diskgroup|controller|server|processor|memory|iraid|
 power|cooling|network|storage|fs|raidsyncstatus|env_hw|vlan

Usage: odaadmcli show - Shows disk, diskgroup, controller, server, processor,
 memory, iraid, power, cooling, network,
 storage, fs, raidsyncstatus, env_hw
 odaadmcli manage - Manages the OAK repository, diagcollect etc.,
 odaadmcli stordiag - Run storage diagnostic tool on this Node
 odaadmcli power - Power on|off|status disk
 odaadmcli expand - Expand storage

Hmmm, looks strange as there is no command CREATE 😯
I can SHOW the VLANs but it looks like I can’t CREATE them… Let’s have a look to the online documentation (here)

Screen Shot 2018-02-20 at 09.16.48

It looks like the CREATE VLAN command finally exist. A good advice to remember here is that even the inline help of ODACLI and ODAADMCLI are quite good, it is still a good practice to have a look to the online documentation and especially the chapters 14 and 15.

The good news here is that the help for the CREATE command does exist 😉

[[email protected] ~]# odaadmcli create vlan -h
odaadmcli create vlan <vlan_name> -vlanid <vlanid> -if <interface> -node <node_num> -setuptype <type> -ip <ip address> -netmask <netmask> -gateway <gateway>, Where:
	 vlan                  -  VLAN name (unique per Node)
	 vlanid                -  Identifies the ID[valid range: 2 to 4094] to which the VLAN belongs to (unique per Node)
	 interface             -  Interface on which the VLAN is to be created [Valid interfaces are btbond1]
	 node                  -  Node number < 0 >
	 setuptype             -  the type of the VLAN setup for [Valid value are: management, database, dataguard, backup, private and other]
	 ip                    -  IP address for the VLAN
	 netmask               -  Netmask address for the VLAN
	 gateway               -  Gateway address for the VLAN

I guess it’s now time to try to create a new VLAN.

[[email protected] ~]# odaadmcli create vlan vlan55-backup -vlanid 55 -if btbond1 -node 0 -setuptype backup -ip 192.168.55.10 -netmask 255.255.255.0 -gateway 192.168.55.1

Created Vlan : vlan55-backup

Note that even on a single node ODA (S or M) you must provide the node number. Otherwise you will get the following error message

ERROR : -node is not available

Let check our newly created VLAN:

[[email protected] network-scripts]# odaadmcli show vlan
	NAME                     ID    INTERFACE   CONFIG_TYPE IP_ADDRESS      NETMASK         GATEWAY         NODENUM
	vlan55-backup            55    btbond1     backup      192.168.55.10   255.255.255.0   192.168.54.1    0

Of course in /etc/sysconfig/network-scripts we will find the corresponding IFCFG file:

[[email protected] network-scripts]# ls -lrt
total 248
-rw-r--r--. 1 root root 29853 Apr 12  2016 network-functions-ipv6
-rw-r--r--. 1 root root 14731 Apr 12  2016 network-functions
...
...
...
-rw-r--r--. 1 root root   264 Feb 19 11:40 ifcfg-lo
-rw-r--r--  3 root root   194 Feb 19 12:04 ifcfg-em3
-rw-r--r--  3 root root   194 Feb 19 12:04 ifcfg-em2
-rw-r--r--  3 root root   169 Feb 19 12:04 ifcfg-em1
drwxr-xr-x  2 root root  4096 Feb 19 12:04 backupifcfgFiles
-rw-r--r--. 3 root root   259 Feb 19 12:17 ifcfg-btbond1
-rw-r--r--  1 root root   538 Feb 19 14:43 ifcfg-btbond1.54
-rw-r--r--  1 root root   538 Feb 19 15:31 ifcfg-btbond1.55

Should a VLAN not be required anymore, deleting it works straight forward using DELETE VLAN:

[[email protected] network-scripts]# odaadmcli delete vlan vlan55-backup -node 0

Deleted Vlan : vlan55-backup

Easy isn’t it? 😎

Last but not least, for those who deployed their ODA without VLAN do not worry you won’t need to re-image it. Even if the ODA has been deployed without VLAN during the configure-firstnet you can still create VLAN afterwards.

Enjoy!

2 Comments

  • Eddie says:

    Hello David,

    Thanks for this write up, it was very helpful. The problem we have with the new ODA X7-2HA is that there are only 2 public ports on each server and we need one more port for our backups. There is an additional 1GB port on the system but this pipe isn’t big enough for time backups for us, so we are turning to VLAN as a solution. My question is, can I setup a VLAN for production and a VLAN for backup and allocate the bandwidth needed for each? We want to be able to prioritize traffic as needed but are not sure if this is done on the ODA or switch level.

    Thanks,

    Eddie

  • David says:

    HI there i really like your blogs regarding ODA. i wanted to ask you that: i have Layer 3 and Layer 2 network, so want to use both for public (layer 3) and db (Layer 2) access, i have ODA X7-2 HA. how do i achieve this and what configuration should be done from switch side, i am using fibre cable.

    Thank You

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

David Hueber
David Hueber

Chief Executive Officer (CEO), Principal Consultant