Networking on balenaOS
Introduction
With balenaOS 2.0, connectivity management changed from ConnMan to NetworkManager. NetworkManager allows balenaOS more flexibility when configuring the network setup, and, in conjunction with ModemManager, allows balenaOS to offer first class GSM/Cellular support.
All of the network configuration for balenaOS can be done through files in the boot partition of your device. If you have a freshly downloaded balenaOS .img, you can mount it and inspect the resin-boot or resin-flash (for devices that boot from eMMC) partition. In the boot partition you will find a directory called system-connections.
When editing files for connectivity in the boot partition, make sure you're in the /mnt/boot/system-connections folder so that they persist after reboot.
The system-connections directory consists of a set of connection files—one file per connection. The file balena-sample.ignore is a template or example file which you can copy to create new connections. If you added a WiFi connection from the dashboard when you downloaded your image, you should see its configuration file already created for you under the name resin-wifi. You will notice that there is no file for ethernet, this is because NetworkManager will always set up a default ethernet connection. If you want a specific configuration for the ethernet connection you will need to create a new connection file for it. Most of the allowed options for these connection files can be found in the NetworkManager settings reference
WiFi Setup
If you entered your WiFi SSID and passphrase when you downloaded balenaOS from the dashboard, you should have a file called resin-wifi in the folder /system-connections/ (as mentioned above, this is found in the boot partition of your image).
[connection]
id=balena-wifi
type=wifi
[wifi]
hidden=true
mode=infrastructure
ssid=My Awesome WiFi Ssid
[ipv4]
method=auto
[ipv6]
addr-gen-mode=stable-privacy
method=auto
[wifi-security]
auth-alg=open
key-mgmt=wpa-psk
psk=super_secret_wifi_passwordThis file sets up a simple WiFi connection to a network named My Awesome WiFi Ssid with a WiFi password (psk) of super_secret_wifi_password. If you want to add multiple different WiFi credentials, then simply make a copy of resin-wifi or balena-sample.ignore and change the SSID and psk key values. You can also add the autoconnect-priority integer values in each file to define the priority of each connection when the device has multiple WiFi connections it can connect to.
To set WiFi as the default route to the internet, you need to force eth0 to have a low route metric and set the never-default option. The following ethernet configuration file can be put in the /system-connections/ folder to accomplish this:
Depending on the device, the interface name may not be eth0. To check the correct name, you can run ifconfig from the web terminal found on the device dashboard.
The connection file also defines what kind of security the network expects. If you want to set up a network with more elaborate security requirements, checkout the NetworkManager settings reference page. If you want to connect to an unsecured WiFi network, simply remove the [wifi-security] section.
As an example the below config sets up a WPA2-enterprise WiFi connection:
An example for Eduroam (network for students and universities) is shown in the config below. You'll need to change the listed fields for your educational institution. Contact your educational institution's IT department for the connection specifics.
Ethernet Setup
Out of the box, Ethernet should work without requiring any specific configuration for Network Manager. There are some instances where you may wish to customize your configuration, such as with sharing a network connection from a WiFi access point to another device connected with Ethernet.
Below is an example for sharing a network connection to another device attached by Ethernet:
Ensure to specify the interface-name, for instance eth0.
By default, when an interface is shared, NetworkManager uses 10.42.x.0/24 as the ipv4 subnet for the interface. This can be set manually by specifying the address1 parameter manner within the [ipv4] block and specifying one address; for example: address1=192.168.1.1. Read more
Regulatory Domain
By default, the WiFi regulatory domain for your device is not set until it is connected to a network. If you need your device to connect at first boot to channels that may be restricted in some countries, you can specify the regulatory domain in the config.json file (like the above WiFi settings, this is found in the boot partition). Regulatory domain is set using the country field.
As an example, if you are deploying your device in Great Britain and need it to connect to channel 12 or 13, you would set "country":"GB".
Setting a Static IP
Setting a static IP is possible by adding a few fields to the [ipv4] section of your connection file.
The important bits to take note of here are interface-name=eth0 which indicates which network interface to use and the address1=192.168.1.111/24,192.168.1.1 line which shows we have a static IP of 192.168.1.111 and the network gateway is 192.168.1.1. The final piece is to add the dns=8.8.8.8;8.8.4.4; line, which tell the device to use Google DNS. The set up for a WiFi connection is very much the same, as can be seen in the below example.
Creating a Hotspot
With NetworkManager, setting up your device as a hotspot is as simple as creating a balena-hotspot file in /system-connections/:
This will create a WPA2/rsn hotspot, in case you need WPA protocol replace proto=rsn with proto= in section [wifi-security].
Cellular Modem Setup
For cellular or GSM based connections, balenaOS makes use of ModemManager and is configured in much the same way as WiFi connections are configured. The connection profile can either be specified in your application code (see Changing the Network at Runtime below), or you can add a cellular configuration by adding a connection file to /resin-boot/system-connections in the downloaded balenaOS .img file. In the future, cellular connections will be configurable from the dashboard at time of adding a device and downloading a balenaOS image.
To set up a cellular connection with your device, just drop the below example configuration into a file in the /resin-boot/system-connections/ directory in the .img or on SD card (if you have it flashed with the OS already) and name it something like cellular. Replace the apn= and number= values with your mobile providers APN and PPP dialing number. If your mobile carrier requires a password and username, you will need to add those as well. For a more in depth look at available settings options for GSM, have a look at Table 58. gsm setting in the NetworkManager GSM settings reference.
Known Tested Modems
Check out the list of balena supported modems
Changing the Network at Runtime
Making changes to the networking of a device is extremely dangerous and can lead to a device being unrecoverable, so exercise caution with any of the following.
Often there are times where the WiFi network that the device will be provisioned into is not known beforehand. In these cases the device needs a mechanism to be able to add its own WiFi credentials during some kind of setup stage. For this reason we built the WiFi Connect project.
WiFi Connect enables the device to create a wireless access point called WiFi Connect (also customizable) which serves a captive portal. This allows your end user to connect to the device and add their WiFi credentials. The device will then automatically leave the setup mode and connect to the specified user network.
Under the hood, WiFi Connect is interacting with NetworkManager via its DBUS API. The DBUS API is a useful way to interact with the balenaOS host NetworkManager, and there are DBUS bindings for most languages. Below is a minimal Python example from the NetworkManager examples, which creates a new NetworkManager connection file that can be used to enable connecting to a new WiFi access point.
You will need to install the dbus module in order to run this example (apt-get install python-dbus) and make sure DBUS_SYSTEM_BUS_ADDRESS=unix:path=/host/run/dbus/system_bus_socket is exported in the environment. This ensures that DBUS calls are directed to the system bus that the host OS is listening on rather than the DBUS bus in your container.
The following example will add a new WiFi connection file to your host OS NetworkManager configuration in /etc/NetworkManager/system-connections. It should be noted that you will not see a new file created in resin-boot/system-connections because there isn't a two-way binding. On device boot, the files in resin-boot/system-connections are copied into /mnt/state/root-overlay/etc/NetworkManager/system-connections without removing any files from the destination. That directory is then bind mounted to /etc/NetworkManager/system-connections in the root filesystem on the host.
The following example will print device name and all associated IPv4 addresses for each network device with an IPv4 connection.
For other network setting needs, such as modifying existing network connection files, or listing connection information, please see the NetworkManager examples.
It can be dangerous to install NetworkManager or python-networkmanager in your container, especially if you have INITSYSTEM=on, as systemd will automatically try to start your container NetworkManager and this will take your devices offline. However, you can prevent this behavior by masking the NetworkManager service in your Dockerfile: RUN apt-get update && apt-get install -y network-manager && systemctl mask NetworkManager.service.
The python-networkmanager Debian package depends on NetworkManager. Preferably python-networkmanager should be installed with pip instead of apt-get. That will avoid the issue described in the warning above and additionally the application container size will be kept minimal.
NetworkManager User Scripts
NetworkManager dispatcher scripts are a powerful feature that allows you to customize the behavior of NetworkManager in response to specific network events. These scripts are executed by NetworkManager when events such as interface up or down, carrier changes, IP address changes, and connection status changes occur.
You can put NetworkManager dispatch user scripts in a folder named /dispatcher.d/ on the boot partition. This is similar to how connection profiles can be placed in a folder named /system-connections/ on the same boot partition.
During the device boot process, the files located in /dispatcher.d/ are copied into the /mnt/state/root-overlay/etc/NetworkManager/dispatcher.d directory. After copying, the /mnt/state/root-overlay/etc/NetworkManager/dispatcher.d directory is bind mounted to /etc/NetworkManager/system-connections in the root filesystem.
Following is an example usage of a dispatcher script provided from the official nmcli usage examples:
This specific dispatcher script ensures that when a wired interface is connected, the WiFi will be set to airplane mode. Conversely, when the wired interface is disconnected, WiFi will be turned back on. You may name the script 70-wifi-wired-exclusive.sh and place it into the /dispatcher.d/ directory on the boot partition.
Network Requirements
In order for a balena device to get outside of the local network and connect to the balena API, there are a few core network requirements.
Balena makes use of the following ports:
443TCP - This is the most fundamental requirement - it is used to connect to Cloudlink and the web terminal, and many web endpoints using TLS (https://.)123UDP - For NTP time synchronization.53UDP - For DNS name resolution.
Each of these should work with outward only (and inward once outward connection established) firewall settings. Additionally, the NTP (123) and DNS (53) ports may be blocked if a local NTP and DNS server are provided.
Additionally, you should allowlist the following domains for the relevant ports above:
*.balena-cloud.com
For older devices using *.resin.io, a 301 Moved Permanently redirection to *.balena-cloud.com is in place so they remain compatible. Your systems should be able to handle this redirect so they can connect to the API.
Example:
Google's Public DNS server at 8.8.8.8 is used by default and in addition to DNS servers obtained via DHCP from your local network or service provider (balenaOS may issue queries to multiple DNS servers simultaneously, for the quickest response to be used). If additional DNS servers are configured via DHCP or other means, it is OK for the local network to block requests to 8.8.8.8. To avoid any requests being made to 8.8.8.8 by balenaOS, modify the dnsServers setting in config.json.
Deploying devices in heavily restricted networks, such as behind country-level firewalls, may affect the ability of the device to connect to cloudlink and is not guaranteed to work.
Connecting Behind a Proxy
We use redsocks to connect your device to balena from behind a proxy. There are two ways to enable redsocks proxy. The preferred method is through Supervisor's GET/PATCH /v1/device/host-config endpoints - more info may be found here.
Alternatively, a redsocks.conf file should be added at /mnt/boot/system-proxy/, specifying the needed proxy configuration:
Note that:
File logging (
log = "file:/var/log/redsocks";) is not used. It will lead to permission problems.daemon = offis set, so that redsocks doesn’t fork.local_port = 12345is set, so that the iptables rules and redsocks port match.Available redsocks types include
socks4,socks5,http-connect, andhttp-relay.
Additionally, a /mnt/boot/system-proxy/no_proxy file can be added with a newline-separated list of IPs or subnets to not route through the proxy.
We've published an example here demonstrating how to implement this on devices.
Checking connectivity
You can configure a connectivity check using NetworkManager's builtin feature (see "connectivity section"). In order to configure custom endpoints, the URL, expected response, and check interval can be set in config.json (more information available here). By default, NetworkManager is configured to use balenaCloud to perform the check.
Disable IPv6
In cases where a local network issue is preventing IPv6 traffic from being routed, you can fully disable IPv6 in balenaOS 2.0 with the following commands.
Making changes to the networking of a device is extremely dangerous and can lead to a device being unrecoverable, so exercise caution with any of the following.
To re-enable IPv6 follow the same commands but with set ipv6.method enable.
Last updated
Was this helpful?