How to setup Ethernet network bonding for root NFS mount over PXE boot in Ubuntu 16.04 LTS
up vote
0
down vote
favorite
For my personal minicluster, I successfully setup an Ubuntu server 16.04 LTS PXE network booting system, however, in order to take advantage of the 4-port NICs I have, I want to setup Ethernet bonding.
I have found no relevant resources to Linux kernel version 4.4.0-21-generic, older solutions are ineffective.
networking server nfs pxe network-bonding
add a comment |
up vote
0
down vote
favorite
For my personal minicluster, I successfully setup an Ubuntu server 16.04 LTS PXE network booting system, however, in order to take advantage of the 4-port NICs I have, I want to setup Ethernet bonding.
I have found no relevant resources to Linux kernel version 4.4.0-21-generic, older solutions are ineffective.
networking server nfs pxe network-bonding
add a comment |
up vote
0
down vote
favorite
up vote
0
down vote
favorite
For my personal minicluster, I successfully setup an Ubuntu server 16.04 LTS PXE network booting system, however, in order to take advantage of the 4-port NICs I have, I want to setup Ethernet bonding.
I have found no relevant resources to Linux kernel version 4.4.0-21-generic, older solutions are ineffective.
networking server nfs pxe network-bonding
For my personal minicluster, I successfully setup an Ubuntu server 16.04 LTS PXE network booting system, however, in order to take advantage of the 4-port NICs I have, I want to setup Ethernet bonding.
I have found no relevant resources to Linux kernel version 4.4.0-21-generic, older solutions are ineffective.
networking server nfs pxe network-bonding
networking server nfs pxe network-bonding
asked Dec 1 at 9:51
JavaProphet
1285
1285
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
up vote
0
down vote
accepted
This is a custom solution, a hack really, that I have managed to get working.
In my NFS that the PXE systems are supposed to boot from, Ensure the /etc/initramfs-tools/initramfs.conf
to have no DEVICE
set, and modules
equal to most
.
From there, you're going to want to add a file at /etc/initramfs-tools/scripts/nfs-premount/00_bond
, ensure it has 0755
permissions, that is it is executable.
00_bond
contents:
#!/bin/sh
ip link add bond0 type bond mode 802.3ad
echo "+eth0" > /sys/class/net/bond0/bonding/slaves
echo "+eth1" > /sys/class/net/bond0/bonding/slaves
echo "+eth2" > /sys/class/net/bond0/bonding/slaves
modprobe af_packet
ipconfig -c dhcp -d bond0
This script creates a bond within the initramfs, manually adds the slaves, as we do not have the tools to do that (they could be added to the initramfs, but unnecessary). Add/remove/rename interfaces as needed.
Finally, we load the af_packet
kernel module, and initialize DHCP on our bonded interface. ipconfig
is a program provided by klibc, not to be confused with ifconfig
.
The last configuration of our initramfs comes in at /etc/initramfs-tools/modules
, where we want to add the bonding
module to our initramfs. Simply add a line bonding
to the end of the file, it is generally empty by default.
You can update the initramfs (make sure you are chrooted or in another system booted into the NFS partition) by running update-initramfs -u
. Once this is complete, make sure to update any usage of it from your TFTP server so that your new initramfs is booted remotely.
In DHCP, since we have bypassed DHCP configuration of the network interfaces (assuming pxelinux), and are dependent on the ethX
NIC name format, we want to change some kernel boot parameters. You can change them in the script, however, I find it more reliable to use the old naming scheme, as this is a case where if a NIC changes, it's best we have drop-in replacement naming -- an issue I have run into before.
These are the kernel boot parameters I used for version 4.4.0-21-generic, as entered in my PXELinux configuration:
APPEND root=/dev/nfs initrd=initrd.img-4.4.0-21-generic nfsroot=192.168.0.1:/nfs net.ifnames=0 biosdevname=0 ip=none rw
net.ifnames=0 biosdevname=0
disables the new NIC naming scheme.
ip=none
disables any default network configuration. I had no issues with ip=dhcp
, but it is unnecessary as it will terminate when it sees a network already in place.
The rest are standard NFS mounting parameters.
Once all of this is complete, your NFS server is running, accessible, and a proper Ubuntu install, your DHCP server is prepared, then you should be ready to boot into Ubuntu with an Ethernet bonded NFS root system, from PXE. The mode I chose, 802.3ad
, provides speed and redundancy from disconnections, while there are other modes available that can provide stronger redundancy in exchange for speed. This is particularly useful for this use case, as when we get a sudden disconnection, it's akin to having the root drive pulled mid-operation.
It should be noted that none of the standard Ubuntu networking will have been configured (i.e. /etc/network/interfaces
), and as such, tools like ifup and ifdown may not work until that is setup. Of course, you should be prepared to power cycle if you were to run such a command, unless you boot a persistent ramdisk with a NFS as a non-root, to maintain minimal functionality.
add a comment |
Your Answer
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "89"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2faskubuntu.com%2fquestions%2f1097622%2fhow-to-setup-ethernet-network-bonding-for-root-nfs-mount-over-pxe-boot-in-ubuntu%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
0
down vote
accepted
This is a custom solution, a hack really, that I have managed to get working.
In my NFS that the PXE systems are supposed to boot from, Ensure the /etc/initramfs-tools/initramfs.conf
to have no DEVICE
set, and modules
equal to most
.
From there, you're going to want to add a file at /etc/initramfs-tools/scripts/nfs-premount/00_bond
, ensure it has 0755
permissions, that is it is executable.
00_bond
contents:
#!/bin/sh
ip link add bond0 type bond mode 802.3ad
echo "+eth0" > /sys/class/net/bond0/bonding/slaves
echo "+eth1" > /sys/class/net/bond0/bonding/slaves
echo "+eth2" > /sys/class/net/bond0/bonding/slaves
modprobe af_packet
ipconfig -c dhcp -d bond0
This script creates a bond within the initramfs, manually adds the slaves, as we do not have the tools to do that (they could be added to the initramfs, but unnecessary). Add/remove/rename interfaces as needed.
Finally, we load the af_packet
kernel module, and initialize DHCP on our bonded interface. ipconfig
is a program provided by klibc, not to be confused with ifconfig
.
The last configuration of our initramfs comes in at /etc/initramfs-tools/modules
, where we want to add the bonding
module to our initramfs. Simply add a line bonding
to the end of the file, it is generally empty by default.
You can update the initramfs (make sure you are chrooted or in another system booted into the NFS partition) by running update-initramfs -u
. Once this is complete, make sure to update any usage of it from your TFTP server so that your new initramfs is booted remotely.
In DHCP, since we have bypassed DHCP configuration of the network interfaces (assuming pxelinux), and are dependent on the ethX
NIC name format, we want to change some kernel boot parameters. You can change them in the script, however, I find it more reliable to use the old naming scheme, as this is a case where if a NIC changes, it's best we have drop-in replacement naming -- an issue I have run into before.
These are the kernel boot parameters I used for version 4.4.0-21-generic, as entered in my PXELinux configuration:
APPEND root=/dev/nfs initrd=initrd.img-4.4.0-21-generic nfsroot=192.168.0.1:/nfs net.ifnames=0 biosdevname=0 ip=none rw
net.ifnames=0 biosdevname=0
disables the new NIC naming scheme.
ip=none
disables any default network configuration. I had no issues with ip=dhcp
, but it is unnecessary as it will terminate when it sees a network already in place.
The rest are standard NFS mounting parameters.
Once all of this is complete, your NFS server is running, accessible, and a proper Ubuntu install, your DHCP server is prepared, then you should be ready to boot into Ubuntu with an Ethernet bonded NFS root system, from PXE. The mode I chose, 802.3ad
, provides speed and redundancy from disconnections, while there are other modes available that can provide stronger redundancy in exchange for speed. This is particularly useful for this use case, as when we get a sudden disconnection, it's akin to having the root drive pulled mid-operation.
It should be noted that none of the standard Ubuntu networking will have been configured (i.e. /etc/network/interfaces
), and as such, tools like ifup and ifdown may not work until that is setup. Of course, you should be prepared to power cycle if you were to run such a command, unless you boot a persistent ramdisk with a NFS as a non-root, to maintain minimal functionality.
add a comment |
up vote
0
down vote
accepted
This is a custom solution, a hack really, that I have managed to get working.
In my NFS that the PXE systems are supposed to boot from, Ensure the /etc/initramfs-tools/initramfs.conf
to have no DEVICE
set, and modules
equal to most
.
From there, you're going to want to add a file at /etc/initramfs-tools/scripts/nfs-premount/00_bond
, ensure it has 0755
permissions, that is it is executable.
00_bond
contents:
#!/bin/sh
ip link add bond0 type bond mode 802.3ad
echo "+eth0" > /sys/class/net/bond0/bonding/slaves
echo "+eth1" > /sys/class/net/bond0/bonding/slaves
echo "+eth2" > /sys/class/net/bond0/bonding/slaves
modprobe af_packet
ipconfig -c dhcp -d bond0
This script creates a bond within the initramfs, manually adds the slaves, as we do not have the tools to do that (they could be added to the initramfs, but unnecessary). Add/remove/rename interfaces as needed.
Finally, we load the af_packet
kernel module, and initialize DHCP on our bonded interface. ipconfig
is a program provided by klibc, not to be confused with ifconfig
.
The last configuration of our initramfs comes in at /etc/initramfs-tools/modules
, where we want to add the bonding
module to our initramfs. Simply add a line bonding
to the end of the file, it is generally empty by default.
You can update the initramfs (make sure you are chrooted or in another system booted into the NFS partition) by running update-initramfs -u
. Once this is complete, make sure to update any usage of it from your TFTP server so that your new initramfs is booted remotely.
In DHCP, since we have bypassed DHCP configuration of the network interfaces (assuming pxelinux), and are dependent on the ethX
NIC name format, we want to change some kernel boot parameters. You can change them in the script, however, I find it more reliable to use the old naming scheme, as this is a case where if a NIC changes, it's best we have drop-in replacement naming -- an issue I have run into before.
These are the kernel boot parameters I used for version 4.4.0-21-generic, as entered in my PXELinux configuration:
APPEND root=/dev/nfs initrd=initrd.img-4.4.0-21-generic nfsroot=192.168.0.1:/nfs net.ifnames=0 biosdevname=0 ip=none rw
net.ifnames=0 biosdevname=0
disables the new NIC naming scheme.
ip=none
disables any default network configuration. I had no issues with ip=dhcp
, but it is unnecessary as it will terminate when it sees a network already in place.
The rest are standard NFS mounting parameters.
Once all of this is complete, your NFS server is running, accessible, and a proper Ubuntu install, your DHCP server is prepared, then you should be ready to boot into Ubuntu with an Ethernet bonded NFS root system, from PXE. The mode I chose, 802.3ad
, provides speed and redundancy from disconnections, while there are other modes available that can provide stronger redundancy in exchange for speed. This is particularly useful for this use case, as when we get a sudden disconnection, it's akin to having the root drive pulled mid-operation.
It should be noted that none of the standard Ubuntu networking will have been configured (i.e. /etc/network/interfaces
), and as such, tools like ifup and ifdown may not work until that is setup. Of course, you should be prepared to power cycle if you were to run such a command, unless you boot a persistent ramdisk with a NFS as a non-root, to maintain minimal functionality.
add a comment |
up vote
0
down vote
accepted
up vote
0
down vote
accepted
This is a custom solution, a hack really, that I have managed to get working.
In my NFS that the PXE systems are supposed to boot from, Ensure the /etc/initramfs-tools/initramfs.conf
to have no DEVICE
set, and modules
equal to most
.
From there, you're going to want to add a file at /etc/initramfs-tools/scripts/nfs-premount/00_bond
, ensure it has 0755
permissions, that is it is executable.
00_bond
contents:
#!/bin/sh
ip link add bond0 type bond mode 802.3ad
echo "+eth0" > /sys/class/net/bond0/bonding/slaves
echo "+eth1" > /sys/class/net/bond0/bonding/slaves
echo "+eth2" > /sys/class/net/bond0/bonding/slaves
modprobe af_packet
ipconfig -c dhcp -d bond0
This script creates a bond within the initramfs, manually adds the slaves, as we do not have the tools to do that (they could be added to the initramfs, but unnecessary). Add/remove/rename interfaces as needed.
Finally, we load the af_packet
kernel module, and initialize DHCP on our bonded interface. ipconfig
is a program provided by klibc, not to be confused with ifconfig
.
The last configuration of our initramfs comes in at /etc/initramfs-tools/modules
, where we want to add the bonding
module to our initramfs. Simply add a line bonding
to the end of the file, it is generally empty by default.
You can update the initramfs (make sure you are chrooted or in another system booted into the NFS partition) by running update-initramfs -u
. Once this is complete, make sure to update any usage of it from your TFTP server so that your new initramfs is booted remotely.
In DHCP, since we have bypassed DHCP configuration of the network interfaces (assuming pxelinux), and are dependent on the ethX
NIC name format, we want to change some kernel boot parameters. You can change them in the script, however, I find it more reliable to use the old naming scheme, as this is a case where if a NIC changes, it's best we have drop-in replacement naming -- an issue I have run into before.
These are the kernel boot parameters I used for version 4.4.0-21-generic, as entered in my PXELinux configuration:
APPEND root=/dev/nfs initrd=initrd.img-4.4.0-21-generic nfsroot=192.168.0.1:/nfs net.ifnames=0 biosdevname=0 ip=none rw
net.ifnames=0 biosdevname=0
disables the new NIC naming scheme.
ip=none
disables any default network configuration. I had no issues with ip=dhcp
, but it is unnecessary as it will terminate when it sees a network already in place.
The rest are standard NFS mounting parameters.
Once all of this is complete, your NFS server is running, accessible, and a proper Ubuntu install, your DHCP server is prepared, then you should be ready to boot into Ubuntu with an Ethernet bonded NFS root system, from PXE. The mode I chose, 802.3ad
, provides speed and redundancy from disconnections, while there are other modes available that can provide stronger redundancy in exchange for speed. This is particularly useful for this use case, as when we get a sudden disconnection, it's akin to having the root drive pulled mid-operation.
It should be noted that none of the standard Ubuntu networking will have been configured (i.e. /etc/network/interfaces
), and as such, tools like ifup and ifdown may not work until that is setup. Of course, you should be prepared to power cycle if you were to run such a command, unless you boot a persistent ramdisk with a NFS as a non-root, to maintain minimal functionality.
This is a custom solution, a hack really, that I have managed to get working.
In my NFS that the PXE systems are supposed to boot from, Ensure the /etc/initramfs-tools/initramfs.conf
to have no DEVICE
set, and modules
equal to most
.
From there, you're going to want to add a file at /etc/initramfs-tools/scripts/nfs-premount/00_bond
, ensure it has 0755
permissions, that is it is executable.
00_bond
contents:
#!/bin/sh
ip link add bond0 type bond mode 802.3ad
echo "+eth0" > /sys/class/net/bond0/bonding/slaves
echo "+eth1" > /sys/class/net/bond0/bonding/slaves
echo "+eth2" > /sys/class/net/bond0/bonding/slaves
modprobe af_packet
ipconfig -c dhcp -d bond0
This script creates a bond within the initramfs, manually adds the slaves, as we do not have the tools to do that (they could be added to the initramfs, but unnecessary). Add/remove/rename interfaces as needed.
Finally, we load the af_packet
kernel module, and initialize DHCP on our bonded interface. ipconfig
is a program provided by klibc, not to be confused with ifconfig
.
The last configuration of our initramfs comes in at /etc/initramfs-tools/modules
, where we want to add the bonding
module to our initramfs. Simply add a line bonding
to the end of the file, it is generally empty by default.
You can update the initramfs (make sure you are chrooted or in another system booted into the NFS partition) by running update-initramfs -u
. Once this is complete, make sure to update any usage of it from your TFTP server so that your new initramfs is booted remotely.
In DHCP, since we have bypassed DHCP configuration of the network interfaces (assuming pxelinux), and are dependent on the ethX
NIC name format, we want to change some kernel boot parameters. You can change them in the script, however, I find it more reliable to use the old naming scheme, as this is a case where if a NIC changes, it's best we have drop-in replacement naming -- an issue I have run into before.
These are the kernel boot parameters I used for version 4.4.0-21-generic, as entered in my PXELinux configuration:
APPEND root=/dev/nfs initrd=initrd.img-4.4.0-21-generic nfsroot=192.168.0.1:/nfs net.ifnames=0 biosdevname=0 ip=none rw
net.ifnames=0 biosdevname=0
disables the new NIC naming scheme.
ip=none
disables any default network configuration. I had no issues with ip=dhcp
, but it is unnecessary as it will terminate when it sees a network already in place.
The rest are standard NFS mounting parameters.
Once all of this is complete, your NFS server is running, accessible, and a proper Ubuntu install, your DHCP server is prepared, then you should be ready to boot into Ubuntu with an Ethernet bonded NFS root system, from PXE. The mode I chose, 802.3ad
, provides speed and redundancy from disconnections, while there are other modes available that can provide stronger redundancy in exchange for speed. This is particularly useful for this use case, as when we get a sudden disconnection, it's akin to having the root drive pulled mid-operation.
It should be noted that none of the standard Ubuntu networking will have been configured (i.e. /etc/network/interfaces
), and as such, tools like ifup and ifdown may not work until that is setup. Of course, you should be prepared to power cycle if you were to run such a command, unless you boot a persistent ramdisk with a NFS as a non-root, to maintain minimal functionality.
edited Dec 1 at 10:01
answered Dec 1 at 9:51
JavaProphet
1285
1285
add a comment |
add a comment |
Thanks for contributing an answer to Ask Ubuntu!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2faskubuntu.com%2fquestions%2f1097622%2fhow-to-setup-ethernet-network-bonding-for-root-nfs-mount-over-pxe-boot-in-ubuntu%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown