정보공유

How to Firmware update for mellanox Unmanaged infiniband switch via InfiniBand Fabric

CheekyKite 2016. 11. 9.

How to Firmware update for mellanox Unmanaged infiniband switch via InfiniBand Fabric.

(2016.11.09 by Dasandata)


안녕하세요!


비 관리형 (Unmanaged) 인피니밴드 스위치의 펌웨어를 

인피니밴드 아답터 (InfiniBand Fabric) 를 통해 업그레이드 하는 방법을 정리해 보았습니다.


자세한 내용은 아래 링크의 사용자 설명서를 참조 부탁 드립니다.

For details refer to Manual : http://www.mellanox.com/pdf/MFT/MFT_user_manual.pdf


감사합니다.






Script started on Tue 08 Nov 2016 10:30:55 PM KST

root@ubuntu:~# ibstat

CA 'mlx4_0'

        CA type: MT4099

        Number of ports: 1

        Firmware version: 2.36.5000

        Hardware version: 1

        Node GUID: 0x7cfe900300a6f870

        System image GUID: 0x7cfe900300a6f873

        Port 1:

                State: Initializing

                Physical state: LinkUp

                Rate: 40

                Base lid: 0

                LMC: 0

                SM lid: 0

                Capability mask: 0x02514868

                Port GUID: 0x7cfe900300a6f871

                Link layer: InfiniBand



root@ubuntu:~# /etc/init.d/opensmd status

● opensmd.service - LSB: Manage OpenSM

   Loaded: loaded (/etc/init.d/opensmd; bad; vendor preset: enabled)

   Active: inactive (dead)

     Docs: man:systemd-sysv-generator(8)




root@ubuntu:~# /etc/init.d/opensmd start

[ ok ] Starting opensmd (via systemctl): opensmd.service.




root@ubuntu:~# ibstat

CA 'mlx4_0'

        CA type: MT4099

        Number of ports: 1

        Firmware version: 2.36.5000

        Hardware version: 1

        Node GUID: 0x7cfe900300a6f870

        System image GUID: 0x7cfe900300a6f873

        Port 1:

                State: Active

                Physical state: LinkUp

                Rate: 40

                Base lid: 3

                LMC: 0

                SM lid: 3

                Capability mask: 0x0251486a

                Port GUID: 0x7cfe900300a6f871

                Link layer: InfiniBand




root@ubuntu:~# iblinkinfo

Switch: 0x0002c9020042dc88 Infiniscale-IV Mellanox Technologies:

           4    1[  ] ==( 4X          10.0 Gbps Active/  LinkUp)==>       3    1[  ] "ubuntu HCA-1" ( )

           4    2[  ] ==(                Down/ Polling)==>             [  ] "" ( )

           4    3[  ] ==(                Down/ Polling)==>             [  ] "" ( )

           4    4[  ] ==(                Down/ Polling)==>             [  ] "" ( )

           4    5[  ] ==(                Down/ Polling)==>             [  ] "" ( )

           4    6[  ] ==(                Down/ Polling)==>             [  ] "" ( )

           4    7[  ] ==(                Down/ Polling)==>             [  ] "" ( )

           4    8[  ] ==(                Down/ Polling)==>             [  ] "" ( )

CA: ubuntu HCA-1:

      0x7cfe900300a6f871      3    1[  ] ==( 4X          10.0 Gbps Active/  LinkUp)==>       4    1[  ] "Infiniscale-IV Mellanox Technologies" ( )





root@ubuntu:~# ibhosts

Ca      : 0x7cfe900300a6f870 ports 1 "ubuntu HCA-1"




root@ubuntu:~# cd /home/user/

root@ubuntu:/home/user# ls -l

total 438456

-rw-r--r-- 1 root root   4194304 Nov  8 18:41 fw-ConnectIB-rel-10_10_5020-MCB194A-FCA_A1.bin

-rwxr-xr-x 1 user user    783980 Oct 24 20:43 fw-ConnectX3-rel-2_36_5000-079DJ3-FlexBoot-3.4.718.bin

-rw-r--r-- 1 root root   1501996 Nov  8 18:41 fw-SX-rel-9_2_8000-MSX6005F_Ax.bin

-rw-r--r-- 1 user user 221511680 Oct 24 18:39 MLNX_OFED_LINUX-3.4-1.0.0.0-ubuntu15.10-x86_64.tar

-rw-r--r-- 1 user user 220979200 Oct 24 18:43 MLNX_OFED_LINUX-3.4-1.0.0.0-ubuntu16.04-x86_64.tar




root@ubuntu:/home/user# ls -l /dev/ms*

ls: cannot access '/dev/ms*': No such file or directory




root@ubuntu:/home/user# which mst

/usr/bin/mst




root@ubuntu:/home/user# mst version

mst, mft 4.5.0-31, built on Sep 07 2016, 17:23:34. Git SHA Hash: a610804




root@ubuntu:/home/user# mst start

Starting MST (Mellanox Software Tools) driver set

Loading MST PCI module - Success

Loading MST PCI configuration module - Success

Create devices





root@ubuntu:/home/user# ls -l /dev/mst/

total 0

crw------- 1 root root 241, 0 Nov  8 22:34 mt4099_pciconf0

crw------- 1 root root 242, 0 Nov  8 22:34 mt4099_pci_cr0




root@ubuntu:/home/user# mst ib add

-I- Discovering the fabric - Running: ibdiagnet -skip all

-I- Added 2 in-band devices




root@ubuntu:/home/user# ls -l /dev/mst/

total 0

-rw-r--r-- 1 root root      0 Nov  8 22:34 CA_MT4099_ubuntu_HCA-1_lid-0x0003

crw------- 1 root root 241, 0 Nov  8 22:34 mt4099_pciconf0

crw------- 1 root root 242, 0 Nov  8 22:34 mt4099_pci_cr0

-rw-r--r-- 1 root root      0 Nov  8 22:34 SW_MT48438_0002c9020042dc88_lid-0x0004




root@ubuntu:/home/user# mst status

MST modules:

------------

    MST PCI module loaded

    MST PCI configuration module loaded


MST devices:

------------

/dev/mst/mt4099_pciconf0         - PCI configuration cycles access.

                                   domain:bus:dev.fn=0000:01:00.0 addr.reg=88 data.reg=92

                                   Chip revision is: 01

/dev/mst/mt4099_pci_cr0          - PCI direct access.

                                   domain:bus:dev.fn=0000:01:00.0 bar=0xdf100000 size=0x100000

                                   Chip revision is: 01


Inband devices:

-------------------

/dev/mst/CA_MT4099_ubuntu_HCA-1_lid-0x0003

/dev/mst/SW_MT48438_0002c9020042dc88_lid-0x0004





root@ubuntu:/home/user# mlxfwmanager -d /dev/mst/SW_MT48438_0002c9020042dc88_lid-0x0004

Querying Mellanox devices firmware ...


Device #1:

----------


  Device Type:      InfiniScaleIV

  Part Number:      MIS5022Q_A1-A2

  Description:      InfiniScale IV QDR InfiniBand Switch; 8 QSFP ports; Unmanaged; No FRUs

  PSID:             MT_0FB0110003

  PCI Device Name:  /dev/mst/SW_MT48438_0002c9020042dc88_lid-0x0004

  Port1 MAC:        N/A

  Port1 GUID:       N/A

  Port2 MAC:        N/A

  Port2 GUID:       N/A

  Versions:         Current        Available

     FW             7.4.0000       N/A


  Status:           No matching image found




root@ubuntu:/home/user# pwd

/home/user

root@ubuntu:/home/user# ls -l

total 438992

-rw-r--r-- 1 root root   4194304 Nov  8 18:41 fw-ConnectIB-rel-10_10_5020-MCB194A-FCA_A1.bin

-rwxr-xr-x 1 user user    783980 Nov  8 22:42 fw-ConnectX3-rel-2_36_5000-079DJ3-FlexBoot-3.4.718.bin

-rw-r--r-- 1 root root    394188 Jul 21  2013 fw-IS4-rel-7_4_3000-MIS5022Q_A1-A5.bin

-rwxr-xr-x 1 root root    150265 Nov  8 22:45 fw-IS4-rel-7_4_3000-MIS5022Q_A1-A5.bin.zip

-rw-r--r-- 1 root root   1501996 Nov  8 18:41 fw-SX-rel-9_2_8000-MSX6005F_Ax.bin

-rw-r--r-- 1 user user 221511680 Oct 24 18:39 MLNX_OFED_LINUX-3.4-1.0.0.0-ubuntu15.10-x86_64.tar

-rw-r--r-- 1 user user 220979200 Oct 24 18:43 MLNX_OFED_LINUX-3.4-1.0.0.0-ubuntu16.04-x86_64.tar

root@ubuntu:/home/user#





root@ubuntu:/home/user# mlxfwmanager -d /dev/mst/SW_MT48438_0002c9020042dc88_lid-0x0004

Querying Mellanox devices firmware ...


Device #1:

----------


  Device Type:      InfiniScaleIV

  Part Number:      MIS5022Q_A1-A2

  Description:      InfiniScale IV QDR InfiniBand Switch; 8 QSFP ports; Unmanaged; No FRUs

  PSID:             MT_0FB0110003

  PCI Device Name:  /dev/mst/SW_MT48438_0002c9020042dc88_lid-0x0004

  Port1 MAC:        N/A

  Port1 GUID:       N/A

  Port2 MAC:        N/A

  Port2 GUID:       N/A

  Versions:         Current        Available

     FW             7.4.0000       7.4.3000


  Status:           Update required


---------

Found 1 device(s) requiring firmware update. Please use -u flag to perform the update.





root@ubuntu:/home/user# mlxfwmanager -u  -d /dev/mst/SW_MT48438_0002c9020042dc88_lid-0x0004

Querying Mellanox devices firmware ...


Device #1:

----------


  Device Type:      InfiniScaleIV

  Part Number:      MIS5022Q_A1-A2

  Description:      InfiniScale IV QDR InfiniBand Switch; 8 QSFP ports; Unmanaged; No FRUs

  PSID:             MT_0FB0110003

  PCI Device Name:  /dev/mst/SW_MT48438_0002c9020042dc88_lid-0x0004

  Port1 MAC:        N/A

  Port1 GUID:       N/A

  Port2 MAC:        N/A

  Port2 GUID:       N/A

  Versions:         Current        Available

     FW             7.4.0000       7.4.3000


  Status:           Update required


---------

Found 1 device(s) requiring firmware update...


Perform FW update? [y/N]: y

Device #1: Updating FW ... Done


Restart needed for updates to take effect.





root@ubuntu:/home/user# mlxfwmanager -d /dev/mst/SW_MT48438_0002c9020042dc88_lid-0x0004

Querying Mellanox devices firmware ...


Device #1:

----------


  Device Type:      InfiniScaleIV

  Part Number:      MIS5022Q_A1-A5

  Description:      InfiniScale IV QDR InfiniBand Switch; 8 QSFP ports; Unmanaged; No FRUs

  PSID:             MT_0FB0110003

  PCI Device Name:  /dev/mst/SW_MT48438_0002c9020042dc88_lid-0x0004

  Port1 MAC:        N/A

  Port1 GUID:       N/A

  Port2 MAC:        N/A

  Port2 GUID:       N/A

  Versions:         Current        Available

     FW             7.4.3000       7.4.3000


  Status:           Up to date




root@ubuntu:/home/user# ibhosts

Ca      : 0x7cfe900300a6f870 ports 1 "ubuntu HCA-1"





root@ubuntu:/home/user# ibstat

CA 'mlx4_0'

        CA type: MT4099

        Number of ports: 1

        Firmware version: 2.36.5000

        Hardware version: 1

        Node GUID: 0x7cfe900300a6f870

        System image GUID: 0x7cfe900300a6f873

        Port 1:

                State: Active

                Physical state: LinkUp

                Rate: 40

                Base lid: 3

                LMC: 0

                SM lid: 3

                Capability mask: 0x0251486a

                Port GUID: 0x7cfe900300a6f871

                Link layer: InfiniBand

root@ubuntu:/home/user#

root@ubuntu:/home/user# exit


Script done on Tue 08 Nov 2016 10:48:18 PM KST

댓글