Am 06.11.2014 um 17:26 schrieb Jiri Pirko:
Thu, Nov 06, 2014 at 05:18:18PM CET, ingo.brand(a)webbilling.de wrote:
> Am 06.11.2014 um 17:03 schrieb Jiri Pirko:
>> Thu, Nov 06, 2014 at 04:26:36PM CET, ingo.brand(a)webbilling.de wrote:
>>> Hello,
>>>
>>> I am currently setting up a high available NFS server cluster with 2 nodes
>>> using pacemaker + corosync + drbd + flashcache.
>>> The two nodes (media1 and media2) have 4 1G copper nics each.
>>> I formed 2 teams:
>>> team0: LACP with em1 and em2 connected to a switch for NFS services
>>> team1: roundrobin with em3 and em4 and MTU 9000 directly connected with
>>> copper patch cables for drbd sync
>>> During my testing everything was working as expected. Then I started to
>>> tranfer about 20TB from an old server to the new cluster. The copy process
>>> was fine for about 32 hours. Then corosync and drbd started to report errors
>>> on the roundrobin team1 interface (see logs at the end of this email).
>>> The drbd sync and corosync started to flap between states for several hours
>>> on team1.
>>> I checked the error counters with ifconfig and found a huge amount off TX
>>> dropped packets on team1 of the drbd slave node media2.
>>> There were NO other error counters raising but team1 on media2.
>>> To make this a bit more clear:
>>> - team1 on media2 (the drbd slave) showed raising TX dropped packets
>>> - em3 and em4 on media2 (the physical interfaces of team1) did not show any
>>> raising error counters.
>>> - team1 on media1 (the drbd master) did not show any raising error counters.
>>> - em3 and em4 on media1 (the physical interfaces of team1) did not show any
>>> raising error counters.
>>> I then thought maybe one of the two patch cables between the two machines
was
>>> not 100% ok. So I removed em4 from team1 on both machines:
>>> "teamdctl team1 port remove em4"
>>> The error counters stopped raising.
>>> I thought: "Yeah! That cable must be broken!"
>>> But just to make sure I really found the problem I re-enabled em4 on both
>>> nodes:
>>> "teamdctl team1 port add em4"
>>> Ok, now both team1 interfaces had 2 links up again and the error counter
>>> restarted to raise again.
>>> I then stopped em3 on both nodes to force all traffic through em4:
>>> "teamdctl team1 port remove em3"
>>> And guess what? The error counters on team1 stopped raising!
>>> So in short:
>>> If I remove either one of the two physical interfaces from team1 everything
>>> is working without any errors. As soon as I enable both physical interfaces
>>> the error counters start to raise.
>>> After these tests I rebooted the slave node (media2) and added both ports to
>>> team1 again (Have you tried turning it off and on again...).
>>> After doing this, I now see TX dropped packets on the not yet rebooted
media1
>>> team1 interface.
>>> Why do I only see these errors on the team1 interface and not on em3 and
em4?
>>>
>>> Currently the copy process of 20TB is still running. I think that if I
reboot
>>> media1 everything will work as expected again for some time because it
worked
>>> during my initial tests.
>>> But I do think that I hit a bug in the teaming driver and the raising error
>>> counters will come back.
>>>
>>> Could anybody help?
Hmm, I went over the results, do not see anything suspicious.
Are you able to run systemtap scripts on the machine?
I can install systemtap and run some scripts. Please provide me some
more information.
>> Interesting. Could you do "teamdctl team1 s d" when
the issue happens
>> and post me the result?
> I just re-enabled port em4 on both machines and the issue happens again. Here
> are your requested results:
> media1 ~]# teamdctl team1 s d
> {
> "ports": {
> "em3": {
> "ifinfo": {
> "dev_addr": "b8:2a:72:d5:6c:e6",
> "dev_addr_len": 6,
> "ifindex": 4,
> "ifname": "em3"
> },
> "link": {
> "duplex": "full",
> "speed": 1000,
> "up": true
> },
> "link_watches": {
> "list": {
> "link_watch_0": {
> "delay_down": 0,
> "delay_up": 0,
> "name": "ethtool",
> "up": true
> }
> },
> "up": true
> }
> },
> "em4": {
> "ifinfo": {
> "dev_addr": "b8:2a:72:d5:6c:e6",
> "dev_addr_len": 6,
> "ifindex": 5,
> "ifname": "em4"
> },
> "link": {
> "duplex": "full",
> "speed": 1000,
> "up": true
> },
> "link_watches": {
> "list": {
> "link_watch_0": {
> "delay_down": 0,
> "delay_up": 0,
> "name": "ethtool",
> "up": true
> }
> },
> "up": true
> }
> }
> },
> "setup": {
> "daemonized": false,
> "dbus_enabled": true,
> "debug_level": 0,
> "kernel_team_mode_name": "roundrobin",
> "pid": 3107,
> "pid_file": "/var/run/teamd/team1.pid",
> "runner_name": "roundrobin",
> "zmq_enabled": false
> },
> "team_device": {
> "ifinfo": {
> "dev_addr": "b8:2a:72:d5:6c:e6",
> "dev_addr_len": 6,
> "ifindex": 7,
> "ifname": "team1"
> }
> }
> }
>
>
> media2 ~]# teamdctl team1 s d
> {
> "ports": {
> "em3": {
> "ifinfo": {
> "dev_addr": "b8:2a:72:d5:97:18",
> "dev_addr_len": 6,
> "ifindex": 4,
> "ifname": "em3"
> },
> "link": {
> "duplex": "full",
> "speed": 1000,
> "up": true
> },
> "link_watches": {
> "list": {
> "link_watch_0": {
> "delay_down": 0,
> "delay_up": 0,
> "name": "ethtool",
> "up": true
> }
> },
> "up": true
> }
> },
> "em4": {
> "ifinfo": {
> "dev_addr": "b8:2a:72:d5:97:18",
> "dev_addr_len": 6,
> "ifindex": 5,
> "ifname": "em4"
> },
> "link": {
> "duplex": "full",
> "speed": 1000,
> "up": true
> },
> "link_watches": {
> "list": {
> "link_watch_0": {
> "delay_down": 0,
> "delay_up": 0,
> "name": "ethtool",
> "up": true
> }
> },
> "up": true
> }
> }
> },
> "setup": {
> "daemonized": false,
> "dbus_enabled": true,
> "debug_level": 0,
> "kernel_team_mode_name": "roundrobin",
> "pid": 3084,
> "pid_file": "/var/run/teamd/team1.pid",
> "runner_name": "roundrobin",
> "zmq_enabled": false
> },
> "team_device": {
> "ifinfo": {
> "dev_addr": "b8:2a:72:d5:97:18",
> "dev_addr_len": 6,
> "ifindex": 7,
> "ifname": "team1"
> }
> }
> }
>
>
>
>> Also you can try random runner instead of roundrobin. The resulting
>> behaviour is almost the same. Maybe the issue won't happen there.
> I do believe that if I "restart" the roundrobin runner on media1, the
problem
> is gone anyway. So for the moment I'll keep roundrobin just to keep the
> failure environment in place in case you need some more data.
>> Also, can you please post me output of:
>> ethtool -i em3
> media1 ~]# ethtool -i em3
> driver: tg3
> version: 3.136
> firmware-version: FFV7.8.16 bc 5720-v1.32
> bus-info: 0000:02:00.0
> supports-statistics: yes
> supports-test: yes
> supports-eeprom-access: yes
> supports-register-dump: yes
> supports-priv-flags: no
>
> media2 ~]# ethtool -i em3
> driver: tg3
> version: 3.136
> firmware-version: FFV7.8.16 bc 5720-v1.32
> bus-info: 0000:02:00.0
> supports-statistics: yes
> supports-test: yes
> supports-eeprom-access: yes
> supports-register-dump: yes
> supports-priv-flags: no
>
>> ethtool em3
> media1 ~]# ethtool em3
> Settings for em3:
> Supported ports: [ TP ]
> Supported link modes: 10baseT/Half 10baseT/Full
> 100baseT/Half 100baseT/Full
> 1000baseT/Half 1000baseT/Full
> Supported pause frame use: No
> Supports auto-negotiation: Yes
> Advertised link modes: 10baseT/Half 10baseT/Full
> 100baseT/Half 100baseT/Full
> 1000baseT/Half 1000baseT/Full
> Advertised pause frame use: Symmetric
> Advertised auto-negotiation: Yes
> Link partner advertised link modes: 10baseT/Half 10baseT/Full
> 100baseT/Half 100baseT/Full
> 1000baseT/Half 1000baseT/Full
> Link partner advertised pause frame use: Symmetric
> Link partner advertised auto-negotiation: Yes
> Speed: 1000Mb/s
> Duplex: Full
> Port: Twisted Pair
> PHYAD: 1
> Transceiver: internal
> Auto-negotiation: on
> MDI-X: off
> Supports Wake-on: g
> Wake-on: d
> Current message level: 0x000000ff (255)
> drv probe link timer ifdown ifup rx_err tx_err
> Link detected: yes
>
> media2 ~]# ethtool em3
> Settings for em3:
> Supported ports: [ TP ]
> Supported link modes: 10baseT/Half 10baseT/Full
> 100baseT/Half 100baseT/Full
> 1000baseT/Half 1000baseT/Full
> Supported pause frame use: No
> Supports auto-negotiation: Yes
> Advertised link modes: 10baseT/Half 10baseT/Full
> 100baseT/Half 100baseT/Full
> 1000baseT/Half 1000baseT/Full
> Advertised pause frame use: Symmetric
> Advertised auto-negotiation: Yes
> Link partner advertised link modes: 10baseT/Half 10baseT/Full
> 100baseT/Half 100baseT/Full
> 1000baseT/Half 1000baseT/Full
> Link partner advertised pause frame use: Symmetric
> Link partner advertised auto-negotiation: Yes
> Speed: 1000Mb/s
> Duplex: Full
> Port: Twisted Pair
> PHYAD: 1
> Transceiver: internal
> Auto-negotiation: on
> MDI-X: on
> Supports Wake-on: g
> Wake-on: d
> Current message level: 0x000000ff (255)
> drv probe link timer ifdown ifup rx_err tx_err
> Link detected: yes
>
>> ethtool -S em3
> media1 ~]# ethtool -S em3
> NIC statistics:
> rx_octets: 26853394715
> rx_fragments: 0
> rx_ucast_packets: 227879267
> rx_mcast_packets: 84
> rx_bcast_packets: 6
> rx_fcs_errors: 0
> rx_align_errors: 0
> rx_xon_pause_rcvd: 0
> rx_xoff_pause_rcvd: 0
> rx_mac_ctrl_rcvd: 0
> rx_xoff_entered: 0
> rx_frame_too_long_errors: 0
> rx_jabbers: 0
> rx_undersize_packets: 0
> rx_in_length_errors: 0
> rx_out_length_errors: 0
> rx_64_or_less_octet_packets: 0
> rx_65_to_127_octet_packets: 0
> rx_128_to_255_octet_packets: 0
> rx_256_to_511_octet_packets: 0
> rx_512_to_1023_octet_packets: 0
> rx_1024_to_1522_octet_packets: 0
> rx_1523_to_2047_octet_packets: 0
> rx_2048_to_4095_octet_packets: 0
> rx_4096_to_8191_octet_packets: 0
> rx_8192_to_9022_octet_packets: 0
> tx_octets: 1179249531811
> tx_collisions: 0
> tx_xon_sent: 0
> tx_xoff_sent: 0
> tx_flow_control: 0
> tx_mac_errors: 0
> tx_single_collisions: 0
> tx_mult_collisions: 0
> tx_deferred: 0
> tx_excessive_collisions: 0
> tx_late_collisions: 0
> tx_collide_2times: 0
> tx_collide_3times: 0
> tx_collide_4times: 0
> tx_collide_5times: 0
> tx_collide_6times: 0
> tx_collide_7times: 0
> tx_collide_8times: 0
> tx_collide_9times: 0
> tx_collide_10times: 0
> tx_collide_11times: 0
> tx_collide_12times: 0
> tx_collide_13times: 0
> tx_collide_14times: 0
> tx_collide_15times: 0
> tx_ucast_packets: 254815741
> tx_mcast_packets: 8
> tx_bcast_packets: 612
> tx_carrier_sense_errors: 0
> tx_discards: 0
> tx_errors: 0
> dma_writeq_full: 0
> dma_write_prioq_full: 0
> rxbds_empty: 0
> rx_discards: 0
> rx_errors: 0
> rx_threshold_hit: 0
> dma_readq_full: 0
> dma_read_prioq_full: 0
> tx_comp_queue_full: 0
> ring_set_send_prod_index: 0
> ring_status_update: 0
> nic_irqs: 0
> nic_avoided_irqs: 0
> nic_tx_threshold_hit: 0
> mbuf_lwm_thresh_hit: 0
>
> media2 ~]# ethtool -S em3
> NIC statistics:
> rx_octets: 782620145025
> rx_fragments: 0
> rx_ucast_packets: 173893766
> rx_mcast_packets: 0
> rx_bcast_packets: 0
> rx_fcs_errors: 0
> rx_align_errors: 0
> rx_xon_pause_rcvd: 0
> rx_xoff_pause_rcvd: 0
> rx_mac_ctrl_rcvd: 0
> rx_xoff_entered: 0
> rx_frame_too_long_errors: 0
> rx_jabbers: 0
> rx_undersize_packets: 0
> rx_in_length_errors: 0
> rx_out_length_errors: 0
> rx_64_or_less_octet_packets: 0
> rx_65_to_127_octet_packets: 0
> rx_128_to_255_octet_packets: 0
> rx_256_to_511_octet_packets: 0
> rx_512_to_1023_octet_packets: 0
> rx_1024_to_1522_octet_packets: 0
> rx_1523_to_2047_octet_packets: 0
> rx_2048_to_4095_octet_packets: 0
> rx_4096_to_8191_octet_packets: 0
> rx_8192_to_9022_octet_packets: 0
> tx_octets: 17210360858
> tx_collisions: 0
> tx_xon_sent: 0
> tx_xoff_sent: 0
> tx_flow_control: 0
> tx_mac_errors: 0
> tx_single_collisions: 0
> tx_mult_collisions: 0
> tx_deferred: 0
> tx_excessive_collisions: 0
> tx_late_collisions: 0
> tx_collide_2times: 0
> tx_collide_3times: 0
> tx_collide_4times: 0
> tx_collide_5times: 0
> tx_collide_6times: 0
> tx_collide_7times: 0
> tx_collide_8times: 0
> tx_collide_9times: 0
> tx_collide_10times: 0
> tx_collide_11times: 0
> tx_collide_12times: 0
> tx_collide_13times: 0
> tx_collide_14times: 0
> tx_collide_15times: 0
> tx_ucast_packets: 147864937
> tx_mcast_packets: 23
> tx_bcast_packets: 2
> tx_carrier_sense_errors: 0
> tx_discards: 0
> tx_errors: 0
> dma_writeq_full: 0
> dma_write_prioq_full: 0
> rxbds_empty: 0
> rx_discards: 0
> rx_errors: 0
> rx_threshold_hit: 0
> dma_readq_full: 0
> dma_read_prioq_full: 0
> tx_comp_queue_full: 0
> ring_set_send_prod_index: 0
> ring_status_update: 0
> nic_irqs: 0
> nic_avoided_irqs: 0
> nic_tx_threshold_hit: 0
> mbuf_lwm_thresh_hit: 0
>
>> ethtool -i em4
> media1 ~]# ethtool -i em4
> driver: tg3
> version: 3.136
> firmware-version: FFV7.8.16 bc 5720-v1.32
> bus-info: 0000:02:00.1
> supports-statistics: yes
> supports-test: yes
> supports-eeprom-access: yes
> supports-register-dump: yes
> supports-priv-flags: no
>
> media2 ~]# ethtool -i em4
> driver: tg3
> version: 3.136
> firmware-version: FFV7.8.16 bc 5720-v1.32
> bus-info: 0000:02:00.1
> supports-statistics: yes
> supports-test: yes
> supports-eeprom-access: yes
> supports-register-dump: yes
> supports-priv-flags: no
>
>> ethtool em4
> media1 ~]# ethtool em4
> Settings for em4:
> Supported ports: [ TP ]
> Supported link modes: 10baseT/Half 10baseT/Full
> 100baseT/Half 100baseT/Full
> 1000baseT/Half 1000baseT/Full
> Supported pause frame use: No
> Supports auto-negotiation: Yes
> Advertised link modes: 10baseT/Half 10baseT/Full
> 100baseT/Half 100baseT/Full
> 1000baseT/Half 1000baseT/Full
> Advertised pause frame use: Symmetric
> Advertised auto-negotiation: Yes
> Link partner advertised link modes: 10baseT/Half 10baseT/Full
> 100baseT/Half 100baseT/Full
> 1000baseT/Half 1000baseT/Full
> Link partner advertised pause frame use: Symmetric
> Link partner advertised auto-negotiation: Yes
> Speed: 1000Mb/s
> Duplex: Full
> Port: Twisted Pair
> PHYAD: 2
> Transceiver: internal
> Auto-negotiation: on
> MDI-X: off
> Supports Wake-on: g
> Wake-on: d
> Current message level: 0x000000ff (255)
> drv probe link timer ifdown ifup rx_err tx_err
> Link detected: yes
>
> media2 ~]# ethtool em4
> Settings for em4:
> Supported ports: [ TP ]
> Supported link modes: 10baseT/Half 10baseT/Full
> 100baseT/Half 100baseT/Full
> 1000baseT/Half 1000baseT/Full
> Supported pause frame use: No
> Supports auto-negotiation: Yes
> Advertised link modes: 10baseT/Half 10baseT/Full
> 100baseT/Half 100baseT/Full
> 1000baseT/Half 1000baseT/Full
> Advertised pause frame use: Symmetric
> Advertised auto-negotiation: Yes
> Link partner advertised link modes: 10baseT/Half 10baseT/Full
> 100baseT/Half 100baseT/Full
> 1000baseT/Half 1000baseT/Full
> Link partner advertised pause frame use: Symmetric
> Link partner advertised auto-negotiation: Yes
> Speed: 1000Mb/s
> Duplex: Full
> Port: Twisted Pair
> PHYAD: 2
> Transceiver: internal
> Auto-negotiation: on
> MDI-X: on
> Supports Wake-on: g
> Wake-on: d
> Current message level: 0x000000ff (255)
> drv probe link timer ifdown ifup rx_err tx_err
> Link detected: yes
>
>> ethtool -S em4
> media1 ~]# ethtool -S em4
> NIC statistics:
> rx_octets: 246361
> rx_fragments: 0
> rx_ucast_packets: 912
> rx_mcast_packets: 8
> rx_bcast_packets: 0
> rx_fcs_errors: 0
> rx_align_errors: 0
> rx_xon_pause_rcvd: 0
> rx_xoff_pause_rcvd: 0
> rx_mac_ctrl_rcvd: 0
> rx_xoff_entered: 0
> rx_frame_too_long_errors: 0
> rx_jabbers: 0
> rx_undersize_packets: 0
> rx_in_length_errors: 0
> rx_out_length_errors: 0
> rx_64_or_less_octet_packets: 0
> rx_65_to_127_octet_packets: 0
> rx_128_to_255_octet_packets: 0
> rx_256_to_511_octet_packets: 0
> rx_512_to_1023_octet_packets: 0
> rx_1024_to_1522_octet_packets: 0
> rx_1523_to_2047_octet_packets: 0
> rx_2048_to_4095_octet_packets: 0
> rx_4096_to_8191_octet_packets: 0
> rx_8192_to_9022_octet_packets: 0
> tx_octets: 760
> tx_collisions: 0
> tx_xon_sent: 0
> tx_xoff_sent: 0
> tx_flow_control: 0
> tx_mac_errors: 0
> tx_single_collisions: 0
> tx_mult_collisions: 0
> tx_deferred: 0
> tx_excessive_collisions: 0
> tx_late_collisions: 0
> tx_collide_2times: 0
> tx_collide_3times: 0
> tx_collide_4times: 0
> tx_collide_5times: 0
> tx_collide_6times: 0
> tx_collide_7times: 0
> tx_collide_8times: 0
> tx_collide_9times: 0
> tx_collide_10times: 0
> tx_collide_11times: 0
> tx_collide_12times: 0
> tx_collide_13times: 0
> tx_collide_14times: 0
> tx_collide_15times: 0
> tx_ucast_packets: 0
> tx_mcast_packets: 8
> tx_bcast_packets: 0
> tx_carrier_sense_errors: 0
> tx_discards: 0
> tx_errors: 0
> dma_writeq_full: 0
> dma_write_prioq_full: 0
> rxbds_empty: 0
> rx_discards: 0
> rx_errors: 0
> rx_threshold_hit: 0
> dma_readq_full: 0
> dma_read_prioq_full: 0
> tx_comp_queue_full: 0
> ring_set_send_prod_index: 0
> ring_status_update: 0
> nic_irqs: 0
> nic_avoided_irqs: 0
> nic_tx_threshold_hit: 0
> mbuf_lwm_thresh_hit: 0
>
> media2 ~]# ethtool -S em4
> NIC statistics:
> rx_octets: 760
> rx_fragments: 0
> rx_ucast_packets: 0
> rx_mcast_packets: 8
> rx_bcast_packets: 0
> rx_fcs_errors: 0
> rx_align_errors: 0
> rx_xon_pause_rcvd: 0
> rx_xoff_pause_rcvd: 0
> rx_mac_ctrl_rcvd: 0
> rx_xoff_entered: 0
> rx_frame_too_long_errors: 0
> rx_jabbers: 0
> rx_undersize_packets: 0
> rx_in_length_errors: 0
> rx_out_length_errors: 0
> rx_64_or_less_octet_packets: 0
> rx_65_to_127_octet_packets: 0
> rx_128_to_255_octet_packets: 0
> rx_256_to_511_octet_packets: 0
> rx_512_to_1023_octet_packets: 0
> rx_1024_to_1522_octet_packets: 0
> rx_1523_to_2047_octet_packets: 0
> rx_2048_to_4095_octet_packets: 0
> rx_4096_to_8191_octet_packets: 0
> rx_8192_to_9022_octet_packets: 0
> tx_octets: 247887
> tx_collisions: 0
> tx_xon_sent: 0
> tx_xoff_sent: 0
> tx_flow_control: 0
> tx_mac_errors: 0
> tx_single_collisions: 0
> tx_mult_collisions: 0
> tx_deferred: 0
> tx_excessive_collisions: 0
> tx_late_collisions: 0
> tx_collide_2times: 0
> tx_collide_3times: 0
> tx_collide_4times: 0
> tx_collide_5times: 0
> tx_collide_6times: 0
> tx_collide_7times: 0
> tx_collide_8times: 0
> tx_collide_9times: 0
> tx_collide_10times: 0
> tx_collide_11times: 0
> tx_collide_12times: 0
> tx_collide_13times: 0
> tx_collide_14times: 0
> tx_collide_15times: 0
> tx_ucast_packets: 927
> tx_mcast_packets: 8
> tx_bcast_packets: 0
> tx_carrier_sense_errors: 0
> tx_discards: 0
> tx_errors: 0
> dma_writeq_full: 0
> dma_write_prioq_full: 0
> rxbds_empty: 0
> rx_discards: 0
> rx_errors: 0
> rx_threshold_hit: 0
> dma_readq_full: 0
> dma_read_prioq_full: 0
> tx_comp_queue_full: 0
> ring_set_send_prod_index: 0
> ring_status_update: 0
> nic_irqs: 0
> nic_avoided_irqs: 0
> nic_tx_threshold_hit: 0
> mbuf_lwm_thresh_hit: 0
>
>
>> Of course in time the problem happens.
>>
>>
>>> Kind regards
>>> Ingo
>>>
>>>
>>> This is the used network config of team1:
>>> media1:
>>> cat /etc/sysconfig/network-scripts/ifcfg-team1_slave_0
>>> # Generated by parse-kickstart
>>> NAME=team1 slave 0
>>> TEAM_MASTER=team1
>>> DEVICETYPE=TeamPort
>>> DEVICE=em3
>>> ONBOOT=yes
>>> UUID=b7026c5b-e9cd-457c-93d2-a2799361ed90
>>>
>>> cat /etc/sysconfig/network-scripts/ifcfg-team1_slave_1
>>> # Generated by parse-kickstart
>>> NAME=team1 slave 1
>>> TEAM_MASTER=team1
>>> DEVICETYPE=TeamPort
>>> DEVICE=em4
>>> ONBOOT=yes
>>> UUID=fbe5bc56-95f5-4f9e-a5ea-ab6b6f5e5b50
>>>
>>> cat /etc/sysconfig/network-scripts/ifcfg-team1
>>> # Generated by parse-kickstart
>>> UUID=4e0090cd-32d3-4c5b-be23-9f53083da4dd
>>> NAME="Team connection team1"
>>> TEAM_CONFIG="{\"runner\": {\"name\":
\"roundrobin\"}}"
>>> GATEWAY=
>>> IPV6_AUTOCONF=yes
>>> BOOTPROTO=none
>>> DEVICE=team1
>>> MTU=9000
>>> TYPE=Team
>>> ONBOOT=yes
>>> IPV6INIT=yes
>>> DEVICETYPE=Team
>>> IPADDR0=192.168.101.31
>>> PREFIX0=24
>>> DEFROUTE=yes
>>> IPV4_FAILURE_FATAL=no
>>> IPV6_DEFROUTE=yes
>>> IPV6_PEERDNS=yes
>>> IPV6_PEERROUTES=yes
>>> IPV6_FAILURE_FATAL=no
>>>
>>>
>>> media2:
>>> cat /etc/sysconfig/network-scripts/ifcfg-team1_slave_0
>>> # Generated by parse-kickstart
>>> NAME=team1 slave 0
>>> TEAM_MASTER=team1
>>> DEVICETYPE=TeamPort
>>> DEVICE=em3
>>> ONBOOT=yes
>>> UUID=bfd8e77e-7eb8-47ea-ab7e-95286e073202
>>>
>>> cat /etc/sysconfig/network-scripts/ifcfg-team1_slave_1
>>> # Generated by parse-kickstart
>>> NAME=team1 slave 1
>>> TEAM_MASTER=team1
>>> DEVICETYPE=TeamPort
>>> DEVICE=em4
>>> ONBOOT=yes
>>> UUID=93c5dfd8-3b81-4d1e-8ba9-c46f8daceb1a
>>>
>>> cat /etc/sysconfig/network-scripts/ifcfg-team1
>>> # Generated by parse-kickstart
>>> UUID=79c843db-996c-41e4-9ee5-2a8f3da244ed
>>> NAME="Team connection team1"
>>> TEAM_CONFIG="{\"runner\": {\"name\":
\"roundrobin\"}}"
>>> GATEWAY=
>>> IPV6_AUTOCONF=yes
>>> BOOTPROTO=none
>>> DEVICE=team1
>>> MTU=9000
>>> TYPE=Team
>>> ONBOOT=yes
>>> IPV6INIT=yes
>>> DEVICETYPE=Team
>>> IPADDR0=192.168.101.32
>>> PREFIX0=24
>>> DEFROUTE=yes
>>> IPV4_FAILURE_FATAL=no
>>> IPV6_DEFROUTE=yes
>>> IPV6_PEERDNS=yes
>>> IPV6_PEERROUTES=yes
>>> IPV6_FAILURE_FATAL=no
>>>
>>>