# How to install Sonic on Celestica dx010 100g switch [![](https://books.netdev.com.tr/uploads/images/gallery/2022-11/scaled-1680-/image-1667994790388.png)](https://books.netdev.com.tr/uploads/images/gallery/2022-11/image-1667994790388.png) In this document i am going to talk about installation of Sonic Network operating system on celestica dx010 32x100G switches If you do these steps you'll end up with a l2 switch that can forward 100g per port. Keep in mind that Spanning tree is still an issue on Sonic. __Important__ : Make sure you check the Sonic HCL, if your model is not there, don't try to do this installation. ### Upgrading or re-installing onie Onie is the Network operating system loader created by open network computing. It is a software to create zero touch provisioning on open compute / Bare metal switches. there are some binaries supplied by open compute platform, and you can compile or cross compile your version of it. By your version i meant the model of the bare metal switch. before 2016 ONL which is sonic based on, supported powerpc cpu models however with sonic you are stuck with intel and some arm cpu's, no powerpc cpu is supported or can be compiled. #### Compiling onie To compile onie you need to read this documentation and have many headaches. I can't get in to details here because it will use a lot of space but it is a good documentation and pretty straight-forward to do. [Onie build documentation](https://opencomputeproject.github.io/onie/developers/building.html) Basically it will create a folder with many permutations of onie for every device out there. if your device is a Powerpc platform then you'll need to do a cross-compile #### installation or upgrading onie using u-boot After powercycle press any key to stop u-boot process. take backup of current environment using printenv output should look like below, please copy and paste it somwhere safe.
Click for environment details ```=> printenv active=image2 autoload=no backup=image1 baudrate=115200 bootactive=fsload 1000000 $active; bootm 1000074 bootargs=root=dev/ram console=ttyS0,115200 bootbackup=fsload 1000000 $backup; bootm 1000074 bootcmd=run check_boot_reason; run nos_bootcmd; run onie_bootcmd bootdelay=3 bootfile=uImage bootimage1=fsload 1000000 image1; bootm 1000074 bootimage2=fsload 1000000 image2; bootm 1000074 check_boot_reason=if test -n $onie_boot_reason; then setenv onie_bootargs boot_reason=$onie_boot_rea son; run onie_bootcmd; fi; consoledev=ttyS0 dboot=setenv bootargs root=/dev/ram mtdparts=physmap-flash.0:126336k(jffs2),4096k(onie),128k(u-boot- env),512k(u-boot) quiet console=ttyS0,$baudrate $dbootfrom && bootimg $dkaddr $draddr $ddaddr ddaddr=0x0D000000 dhcp_user-class=powerpc-quanta_ly2r-r0_uboot dhcp_vendor-class-identifier=powerpc-quanta_ly2r-r0 diag_bootcmd=run dsdboot || run flash_bootcmd || run dnetboot dkaddr=0x0A000000 dmemerase=mw $dkaddr 0 100; mw $draddr 0 100; mw $ddaddr 0 100; setenv dbootfrom dnetboot=run dmemerase; run dnetcheck; if test -n $dbootfrom && test $dbootfrom = netboot; then run dboot; fi dnetcheck=echo Checking diag from tftp server...; if tftp $dkaddr uImage && tftp $draddr uInitrd2m & & tftp $ddaddr ly2r.dtb; then setenv dbootfrom netboot; fi draddr=0x0B000000 dsdboot=run dmemerase; run dsdcheck; if test -n $dbootfrom && test $dbootfrom = sdboot; then run dbo ot; fi dsdcheck=echo Checking diag from Secure Digital Card...; if mmcinfo && ext2load mmc 0:1 $dkaddr uIma ge && ext2load mmc 0:1 $draddr uInitrd2m && ext2load mmc 0:1 $ddaddr ly2r.dtb; then setenv dbootfrom sdboot; fi ethact=eTSEC1 ethaddr=a8:1e:84:58:67:10 flash_bootcmd=setenv bootargs root=/dev/ram mtdparts=physmap-flash.0:58368k(jffs2),3968k(ramdisk),30 72k(kernel),128k(dtb),60800k(empty),4096k(onie),128k(u-boot-env),512k(u-boot) console=ttyS0,$baudrat e; bootm EBCE0000 EB900000 EBFE0000 hostname=LY2RP2020 hwconfig=usb1:dr_mode=host,phy_type=ulpi,esdhc ipaddr=192.168.2.1 loadaddr=1000000 loads_echo=1 mtddevnum=0 mtdids=nor0=nor0 [0/0] mtdparts=nor0:65536k(jffs2),60800k(open),4096k(ONIE),128k(u-boot-env),512k(u-boot) nfsboot=setenv bootargs root=/dev/nfs rw nfsroot=$serverip:$rootpath ip=$ipaddr:$serverip:$gatewayip:$netmask:$hostname:$netdev:off console=$consoledev,$baudrate $othbootargs;tftp $loadaddr $bootfile;tftp $fdtaddr $fdtfile;bootm $loadaddr - $fdtaddr nos_bootcmd=run bootactive; run bootbackup; run bootimage1; run bootimage2 onie_args=run onie_initargs onie_platformargs onie_bootcmd=echo Loading Open Network Install Environment ...; echo Platform: $onie_platform ; echo Version : $onie_version ; cp.b $onie_start $loadaddr ${onie_sz.b} && run onie_args && bootm ${loadaddr} onie_dropbear_dss_host_key=begin-base64@600@d#AAAAB3NzaC1kc3MAAACBAOCuKIyK2CzC6aUUNVfJLj6SlJ+anzWWUbuqtyn4#oty/BYzbG7rk+9exB/I9SOBMf0n8BHGJOdS+hXVFe8CAGf0fI7AlHF0zTZCN#kZAW70T175b4AZC8zCegABBKv86s0s7OT4qOXLCeYjCM9gMWHRDAYHDfzktY#vBHapERPhIHzAAAAFQDfAxkC7+W/gxwO8BAg9Se08laRBwAAAIEAyjX1GFoX#oqPyEVn/qQ85xXpo3fr3UNT/NmZZGXQFZ+pUnHDOO6OwTZB0XLpPxzQ8WUVJ#Bt9d2tLC5Sw6rCp7odbp0Od9uWOc7ljt7HKtQVuodB/yjt9KbQc69sOY25bC#efY2c0IcZ9LiNbQbH4COR6zGMLzA2XgHQ7uJ7xmV1fsAAACBAJP+ZxxFAehm#H9Ac5HUHYG4ithFYK7l5MHRrA/pEGuqwUeezUKO9B8soCP9r4SoT/vLzUxQ5#TMwyANyi7BRjKnkjrRjJ/r6o9t5PS7YuGm/3G6hOL7UHJoDIDFuznL9NG+f4#GhHF6NElGGnpmATry0+5gTN2KDYlZLoMriGaGBLVAAAAFQCDpjU05iU1Lbci#IvGrJfEo2HQweg==#====# onie_dropbear_rsa_host_key=begin-base64@600@r#AAAAB3NzaC1yc2EAAAADAQABAAAAgQC8AVEAfeuMSVYNPJPULsBYL0KrFOOM#62CmnWNODmX230TZYW/DDv/NSOMUNFUR109MpzEyy8WmNqqNl+h0X3dK/KUy#H4iZhIxXXYaRu8Yywfaf/xXNnCcQqdj5JYmNGlaRggcvkbYhnX2Hus7LrkGP#k0xdkl5erOsUGPmnf5RLNQAAAIAqqn3Qh8MiBi6iOGIwICAEo00U+FufyncX#uNYli7z2JrtAPg2YDdpdozHtjubTcrKLadIPfhEv5G4WTigx6Mxwcw1oHUon#LoBzMCEjULMpyReO5xQcyV0WTLyPncPgqh4wtT7gr2ePkEASwMgcIiXu0tV9#Q81k8UOcMZQz5kc3QQAAAEEA8mO+xK2LBWrmT37yh3c4xbg/Naj60CPxzRKR#HF0w6FzL8BcCAL009z/YHtIJVpoXo321c8DNazZ47hQEoQpv0QAAAEEAxo/Q#1zXuEwsJUqrOE5RZ8KdfsQSEw9I+Pl8pDNBV06MTbnQPgjMr9beD6NfSo1po#uZmbPVzaakzOtyduVCqCJQ==#====# onie_initargs=setenv bootargs quiet console=$consoledev,$baudrate onie_machine=quanta_ly2r onie_machine_rev=0 onie_platform=powerpc-quanta_ly2r-r0 onie_platformargs=setenv bootargs $bootargs serial_num=${serial#} eth_addr=$ethaddr $onie_bootargs $onie_debugargs onie_rescue=setenv onie_boot_reason rescue && boot onie_start=0xefb60000 onie_sz.b=0x00400000 onie_uninstall=setenv onie_boot_reason uninstall && boot onie_update=setenv onie_boot_reason update && boot onie_vendor_id=7244 onie_version=2015.05-v1.2-9d597eb partition=nor0,0 platform=quanta_ly2r ramboot=setenv bootargs root=/dev/ram rw console=$consoledev,$baudrate $othbootargs; tftp $ramdiskaddr $ramdiskfile;tftp $loadaddr $bootfile;tftp $fdtaddr $fdtfile;bootm $loadaddr $ramdiskaddr $fdtaddr rootpath=/opt/nfsroot serial#=QTFCEA7070054 serverip=192.168.2.11 stderr=serial stdin=serial stdout=serial ver=U-Boot 2010.12 (Oct 14 2016 - 16:28:02) - ONIE 2015.05-v1.2-9d597eb Environment size: 5059/8188 bytes```
We are going to change environment as shown and explanied below, Set ip address to eth0/management port of bare metal switch setenv ipaddr 4.3.2.2 setenv netmask 255.255.255.252 Specify tftp server address setenv serverip 4.3.2.1 save changes save Make sure have the same subnet with tftp server After this point all the procedure will be destructive, please keep in mind that you might loose the current boot configuration or configurations within device. specify space in flash setenv start 0xefb60000 setenv sz.b 0x4a0000 upload file to ram tftp onie-quanta_ly2r-r0.bin Disable protection on flash and erase the old onie protect off $start +${sz.b} && erase $start +${sz.b} start flashing cp.b $fileaddr $start ${sz.b} && protect on $start +${sz.b} after this point Onie will be upgraded/installed and procedure will reset the device booting into new onie and it's environment ### Installing sonic ##### Before install Sonic is a buggy software, there are many missing features, and because of that there are many builds for every vendor the list is [here](https://sonic-build.azurewebsites.net/ui/sonic/Pipelines) we are going to use 202111 edition on celestica dx010 ##### installation Onie will boot up and you will be greeted with onie-discovery process, unless you did special dhcp configurations, we will need to stop onie-discovery onie-discover-stop then the ip addressing ip addr add 4.3.2.2/30 dev eth0 ip link set up dev eth0 start installing sonic onie-nos-install tftp://4.3.2.1/sonic-buildimage.broadcom at this point onie installer will do its thing after 5-10 minutes you will be greeted with sonics login screen ##### After install Default user name: admin default pass : YourPaSsWoRd By default sonic names every 100g interface with steps of 4, Ethernet0,Ethernet4,Ethernet8 and there are aliases for you to see which port are you working on admin@sonic:~$ show interfaces status Interface Lanes Speed MTU FEC Alias Vlan Oper Admin Type Asym PFC ----------- --------------- ------- ----- ----- ------- ------ ------ ------- -------------- ---------- Ethernet0 65 10G 9100 N/A Eth1/1 routed down down QSFP+ or later N/A Ethernet1 66 10G 9100 N/A Eth1/2 routed down down QSFP+ or later N/A Ethernet2 67 10G 9100 N/A Eth1/3 routed down down QSFP+ or later N/A Ethernet3 68 10G 9100 N/A Eth1/4 routed down down QSFP+ or later N/A Ethernet4 69 10G 9100 N/A Eth2/1 routed down down QSFP+ or later N/A Ethernet5 70 10G 9100 N/A Eth2/2 routed down down QSFP+ or later N/A Ethernet6 71 10G 9100 N/A Eth2/3 routed down down QSFP+ or later N/A Ethernet7 72 10G 9100 N/A Eth2/4 routed down down QSFP+ or later N/A Ethernet8 73,74,75,76 100G 9100 N/A Eth3 routed down up N/A N/A Ethernet12 77,78,79,80 100G 9100 N/A Eth4 routed down up N/A N/A Ethernet16 33,34,35,36 100G 9100 N/A Eth5 routed down up N/A N/A Ethernet20 37,38,39,40 100G 9100 N/A Eth6 routed down up N/A N/A Ethernet24 41,42,43,44 100G 9100 N/A Eth7 routed down up N/A N/A Ethernet28 45,46,47,48 100G 9100 N/A Eth8 routed down up N/A N/A Ethernet32 49,50,51,52 100G 9100 N/A Eth9 routed down up N/A N/A Ethernet36 53,54,55,56 100G 9100 N/A Eth10 routed down up N/A N/A Ethernet40 57,58,59,60 100G 9100 N/A Eth11 routed down up N/A N/A Ethernet44 61,62,63,64 100G 9100 N/A Eth12 routed down up N/A N/A Ethernet48 81,82,83,84 100G 9100 N/A Eth13 routed down up N/A N/A Ethernet52 85,86,87,88 100G 9100 N/A Eth14 routed down up N/A N/A Ethernet56 89,90,91,92 100G 9100 N/A Eth15 routed down up N/A N/A Ethernet60 93,94,95,96 100G 9100 N/A Eth16 routed down up N/A N/A Ethernet64 97,98,99,100 100G 9100 N/A Eth17 routed down up N/A N/A Ethernet68 101,102,103,104 100G 9100 N/A Eth18 routed down up N/A N/A Ethernet72 105,106,107,108 100G 9100 N/A Eth19 routed down up N/A N/A Ethernet76 109,110,111,112 100G 9100 N/A Eth20 routed down up N/A N/A Ethernet80 1,2,3,4 100G 9100 N/A Eth21 routed down up N/A N/A Ethernet84 5,6,7,8 100G 9100 N/A Eth22 routed down up N/A N/A Ethernet88 9,10,11,12 100G 9100 N/A Eth23 routed down up N/A N/A Ethernet92 13,14,15,16 100G 9100 N/A Eth24 routed down up N/A N/A Ethernet96 17 25G 9100 N/A Eth25/1 routed down down QSFP+ or later N/A Ethernet97 18 25G 9100 N/A Eth25/2 routed down down QSFP+ or later N/A Ethernet98 19 25G 9100 N/A Eth25/3 routed down down QSFP+ or later N/A Ethernet99 20 25G 9100 N/A Eth25/4 routed down down QSFP+ or later N/A Ethernet100 21,22,23,24 100G 9100 N/A Eth26 routed down up N/A N/A Ethernet104 25,26,27,28 100G 9100 N/A Eth27 routed down up N/A N/A Ethernet108 29,30,31,32 40G 9100 N/A Eth28 routed down up QSFP+ or later N/A Ethernet112 113,114,115,116 100G 9100 N/A Eth29 routed down up N/A N/A Ethernet116 117,118,119,120 40G 9100 N/A Eth30 routed down up QSFP+ or later N/A Ethernet120 121,122,123,124 100G 9100 N/A Eth31 routed down up N/A N/A Ethernet124 125,126,127,128 100G 9100 N/A Eth32 routed down up N/A N/A Sonic has some problems, it's not the news, it's a fact. One of them is this installation media doesn't create a running environment, to do that follow the steps below sudo sonic-cfggen --preset l2 -p -H -k Seastone-DX010 -w sudo config reload -y sudo reboot Clear the IP addresses from each interface show runningconfiguration interfaces | grep \| | awk -F'"' '{ print $2 }' | awk -F'|' '{ print "sudo config interface ip remove "$1" "$2 }' > /var/tmp/remove-l3-ips bash /var/tmp/remove-l3-ips rm -f /var/tmp/remove-l3-ips ### Configure sonic #### Layer1 Every port and platform or asic has different abilities or features, for example you can breakout every port on a celestica dx010 how ever an edge-core AS9716-32D will only do breakouts on Ethernet0-20 which is the top row of switch. I was able to work with 3rdparty qsfp modules on sonic however i couldn't find an HCL about qsfps . ##### Breakouts You neeed to do everything written on _after install_ to be able to do breakouts. Supported breakouts are ; - 1x100G[100g] - 1x100G[40g] - 2x50G[50G] - 4x25G[25G] - 4x25G[10G] to break out 1 port to 4 10g ports sudo config interface breakout Ethernet96 4x25G[10G] -y a normal output should look like below, after this you will need to reboot the switch. admin@sonic:~$ sudo config interface breakout Ethernet96 4x25G[10G] -y Running Breakout Mode : 1x100G[40G] Target Breakout Mode : 4x25G[10G] Ports to be deleted : { "Ethernet96": "100000" } Ports to be added : { "Ethernet96": "25000", "Ethernet97": "25000", "Ethernet98": "25000", "Ethernet99": "25000" } After running Logic to limit the impact Final list of ports to be deleted : { "Ethernet96": "100000" } Final list of ports to be added : { "Ethernet96": "25000", "Ethernet97": "25000", "Ethernet98": "25000", "Ethernet99": "25000" } sonic_yang(6):Note: Below table(s) have no YANG models: SNMP, SNMP_COMMUNITY Below Config can not be verified, It may cause harm to the system {} Do you wish to Continue? [y/N]: y Breakout process got successfully completed. Please note loaded setting will be lost after system reboot. To preserve setting, run `config save`. Dont forget to save :) admin@sonic:~$ sudo config save Existing files will be overwritten, continue? [y/N]: y Running command: /usr/local/bin/sonic-cfggen -d --print-data > /etc/sonic/config_db.json You __will reboot__ the switch so it can show newly emerged ports. After reboot you will have Ethernet 96 to 99, set the speeed to 10g on those interfaces config interface speed Ethernet96 10000 And start the port, config interface startup Ethernet96 Do this for every port you broke and want to work immediately #### Layer2 ##### Creating Vlans here sudo config vlan add 1000 ##### vlans and membership to see the details show vlan brief ##### tagging, untagged ports - tagged sudo config vlan member add 1000 Ethernet48 - untagged sudo config vlan member add -u 1000 Ethernet48 You can use the following commands to edit ports in bulk seq 36 39|xargs -i -t sudo config interface speed Ethernet{} 10000 seq 36 39|xargs -i -t sudo config interface startup Ethernet{} seq 36 39|xargs -i -t sudo config vlan member add -u 1000 Ethernet{} #### Layer3 ##### intervlan routing to add an ip or svi to vlan 1 use the command below admin@sonic:~$ sudo config interface ip add Vlan1000 192.168.1.2/24 ##### routed interfaces as in svi configuration but this time use the port name sudo config interface ip add Ethernet4 192.168.1.2/24