Saturday, September 15, 2012

em0 and em1 interfaces on SRX cluster

Customer asked me this week about em0 and em1 on SRX cluster.  Specifically, he wanted to know what these interfaces are used for, why one of them is down in his configuration, and why they have strange (and seemingly public, non-RFC1918) addressing.

This is from a pair of clustered SRX1400s:

{primary:node0}[edit]
admin@srx1400-node0# run show interfaces terse | match inet
em0.0                   up    up   inet     129.16.0.1/2   
em1.0                   up    up   inet     129.16.0.1/2   
fab0.0                  up    up   inet     30.17.0.200/24 
fab1.0                  up    up   inet     30.18.0.200/24 
fxp0.0                  up    up   inet     192.168.1.150/24
lo0.0                   up    up   inet     192.168.40.3        --> 0/0
lo0.16384               up    up   inet     127.0.0.1           --> 0/0
lo0.16385               up    up   inet   
reth0.0                 up    up   inet     192.168.31.2/24
reth1.0                 up    up   inet     192.168.32.2/24
reth2.0                 up    up   inet     192.168.34.1/24
reth3.0                 up    up   inet     192.168.35.1/24

There's a Juniper KB article that says "Both EM0 and EM1 are internal interfaces that connect between the Routing Engine (RE) and the Control Board (CB).  There is no correlation between EM0/EM1 to any physical interfaces."

The em0 and em1 interfaces correspond to the 2 potential control links between the clustered chassis.  On the SRX1400, ports ge-0/0/10 and ge-0/0/11 on both chassis are dedicated to this function when the chassis are configured into a cluster.  When both sets of ports are cabled together, both control links are up:

{primary:node0}[edit]
admin@srx1400-node0# run show chassis cluster interfaces
Control link status: Up

Control interfaces:
    Index   Interface        Status
    0       em0              Up   
    1       em1              Up   


When the link on ge-0/0/11 is disconnected, em1 goes down:

{primary:node0}[edit]
admin@srx1400-node0# run show chassis cluster interfaces    
Control link status: Up

Control interfaces:
    Index   Interface        Status
    0       em0              Up   
    1       em1              Down  


Control plane redundancy works a little differently on the SRX3K and the SRX5K series, but this is a separate blog post.

As for the strange addressing, it is not unusual on JUNOS platform for the device to use internal-only addressing for intrachassis communication.  These addresses show in the earlier output for the em and fab addressing live inside private, usually-ignored routing instances:

{primary:node0}[edit]
admin@srx1400-node0# run show route private

__juniper_private1__.inet.0: 4 destinations, 5 routes (4 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

30.18.0.0/24       *[Direct/0] 07:36:24
                    > via fab1.0
30.18.0.200/32     *[Local/0] 11:32:26
                      Local via fab1.0
128.0.0.0/2        *[Direct/0] 11:43:39
                    > via em0.0
                    [Direct/0] 11:43:39
                    > via em1.0
129.16.0.1/32      *[Local/0] 11:43:39
                      Local

__juniper_private2__.inet.0: 1 destinations, 1 routes (0 active, 0 holddown, 1 hidden)


These addresses are not reachable from the global routing table:

{primary:node0}[edit]
admin@srx1400-node0# run ping 129.16.0.1
PING 129.16.0.1 (129.16.0.1): 56 data bytes
ping: sendto: No route to host
ping: sendto: No route to host
ping: sendto: No route to host
^C
--- 129.16.0.1 ping statistics ---
3 packets transmitted, 0 packets received, 100% packet loss


And are only reachable from inside the private routing-instance:

{primary:node0}[edit]
admin@srx1400-node0# run ping 129.16.0.1 routing-instance __juniper_private1__   
PING 129.16.0.1 (129.16.0.1): 56 data bytes
64 bytes from 129.16.0.1: icmp_seq=0 ttl=64 time=0.197 ms
64 bytes from 129.16.0.1: icmp_seq=1 ttl=64 time=0.120 ms
64 bytes from 129.16.0.1: icmp_seq=2 ttl=64 time=0.119 ms
^C
--- 129.16.0.1 ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max/stddev = 0.119/0.145/0.197/0.037 ms


No comments:

Post a Comment