Unlike the straightforward installation process we're accustomed to with more mainstream architectures, this task required delving into the nuances of UEFI bootloader configurations and intricately managing the board's boot process.

This blog post is a deep dive into that journey, aiming to demystify the steps needed to achieve a stable and functional Ubuntu setup on this emerging architecture.

📋 Prerequisites

⚠️ Before you begin, your VisionFive 2 board will need to be running the latest firmware. For a guide on how to do this, check out my previous blog post ⚠️

Hardware you will need:

• A MicroSD card (8GB+)

• NVME SSD (installed on the underside of the VisionFive 2 board)

• USB to UART dongle

Files:

• Ubuntu Server Installer (choose the StarFive VisionFive 2 option, then download the "live installer"

✅ Board Setup

Flash the installer to the MicroSD card using Etcher and insert it into the MicroSD slot of the board.

Set the DIP switches to set the Boot Mode to Flash

• RGPIO_1 -> 0

• RGPIO_0 -> 0

Finally, set up the UART to USB connection. I have been using the Raspberry Pi Debug Probe which works great. Connect the UART leads to the board as follows:

• Pin 6 -> GND

• Pin 8 -> RX

• Pin 10 -> TX

Next, you need to connect to the serial terminal. If you are using macOS for example, the serial device will show with a prefix of tty.usbmodem, so you can search for it with the command:

ls /dev | grep tty.usb

Then initialise a connection to it using picocom:

sudo picocom -b 115200 /dev/tty.usbmodem2112402

Now we are ready to install Ubuntu!

👢Booting the Ubuntu Installer

To successfully boot the installer on the StarFive VisionFive 2 board, we must first identify the appropriate MicroSD Card and manually load essential files using U-Boot commands. Unlike more conventional systems where this process might be automated, the Ubuntu Installer for this specific RISC-V architecture requires direct intervention through the serial console. This step is crucial for initiating the installation process.

Firstly, power on the VisionFive board and interrupt the boot process when we see the following:

Hit any key to stop autoboot:  2

We start by listing all available memory cards with mmc list to see what options we have:

StarFive # mmc list
sdio0@16010000: 0
sdio1016020000: 1

In this case, we can see two devices listed; however, only one will be the MicroSD card with the installer on it. We can use the part list command to probe the device for partitions, and from there, determine which device and partition we need.

Let's examine the first device:

StarFive # part list mmc 0
Card did not respond to voltage select! : -110

Any time you see Card did not respond to voltage select! : -110, this generally means the device does not exist, or is not valid. So in this case we need to inspect the device with ID 1:

StarFive # part list mmc 1

Partition Map for MMC device 1  --   Partition Type: EFI

Part    Start LBA    End LBA        Name
    Attributes
    Type GUID
    Partition GUID
  1    0x0000a000    0x0003efff    "EFI System"
    attrs:    0x0000000000000000
    type:    c12a7328-f81f-11d2-ba4b-00a0c93ec93b
    type:    system
    guid:    df2030ee-9281-40ba-8e38-027c9f37917e
  2    0x0003f000    0x0063efff    "Live Media"
    attrs:    0x0000000000000004
    type:    0fc63daf-8483-4772-8e79-3d69d8477de4
    type:    linux
    guid:    4b987a45-c7ad-4804-9048-717d7d37d6b0
  3    0x00000022    0x00000821    "loader1"
    attrs:    0x0000000000000000
    type:    5b193300-fc78-40cd-8002-e86c45580b47
    guid:    ad54bbfb-4527-4681-8f6e-fa7ef471109f
  4    0x00000822    0x00002821    "loader2"
    attrs:    0x0000000000000000
    type:    2e54b353-1271-4842-806f-e436d6af6985
    guid:    3a71d22e-655d-46a6-99db-66c67ea264a1
  5    0x0063f000    0x0e81f7ff    ""
    attrs:    0x0000000000000000
    type:    0fc63daf-8483-4772-8e79-3d69d8477de4
    type:    linux
    guid:    84bc9b76-b57e-4fa4-9367-cef404541690

We are looking for the EFI System partition, which in this case is partition number 1 of device 1

Next, we load a crucial hardware configuration file called the Device Tree Blob (DTB) from the the EFI Partition of the MicroSD Card (mmc 1:1) into the system's memory using the load command as below:

load mmc 1:1 $fdt_addr_r dtb/starfive/jh7110-starfive-visionfive-2-v1.3b.dtb

Following that, we also load the RISC-V Linux kernel, located in an EFI file, into memory using the load command:

load mmc 1:1 $kernel_addr_r EFI/boot/bootriscv64.efi

You will know the load command has run successfully if you see a message in the format of 12345 bytes read in 10 ms.

Finally, we initiate the boot process with the bootefi command:

bootefi $kernel_addr_r $fdt_addr_r

This uses the loaded kernel and hardware configuration file to start up the installer. This sequence of commands essentially prepares the system and directs it to boot up from the specified files located on the memory card.

If successful, after a few seconds you should be greeted with the GRUB bootloader screen:

Hit enter, or wait 30s, to proceed to the installer.

⚙️ Running Through the Installer

The installer can take a few minutes to run through the boot process and can, at times, appear to hang, but just let it run and it will eventually show you the following screen:

From this menu, you can continue working with the installer via the serial console in either rich or basic mode. However, personally I found it easier, and more reliable, to use the SSH method. If you choose this method, you will be provided instructions on how to access the installer over SSH:

Once you SSH in, you will see the standard Ubuntu Server installer program:

The rest of the installer is the same as any other system, so work through it and configure the installation to your needs. Be sure to install Ubuntu on the NVME drive when prompted for the disk configuration - this will likely be the default option.

Once installed, let the device reboot automatically and move on to the final stage of configuring Ubuntu on the StarFive 2 board.

🥾Booting Ubuntu on NVME

Once the board reboots, you will likely be greeted with error messages in the serial console and Ubuntu will not boot automatically from the NVME drive:

Retrieving file: /extlinux/extlinux.conf
Can't set block device
Error reading config file

Don't panic, this is normal! In most systems, the Ubuntu installer will automatically configure UEFI variables to tell the bootloader how to boot into GRUB, however, U-Boot does not allow operating systems to configure these directly. So, we will have to do this manually.

Once the board has hit the error, it should drop you into the U-Boot prompt. From here you can run commands to boot Ubuntu manually:

pci enum
nvme scan
efidebug boot add -b 0001 'Ubuntu' nvme 0:1 /EFI/ubuntu/grubriscv64.efi
efidebug boot order 0001
bootefi bootmgr

We first enumerate the PCI devices on the board with pci enum, then we use nvme scan to identify and prepare any connected NVMe drives for use, allowing the system to recognize and interact with them.

Next, we need to add a UEFI boot entry with efidebug boot add. In this case, we are setting boot entry 1 and pointing it to the NVME drive with nvme 0:1 and finally pointing it to the EFI executable /EFI/ubuntu/grubriscv64.efi.

We then set the boot order with efidebug boot order and then use bootefi bootmgr to boot the system using the UEFI boot manager. This will attempt to boot from the boot entries in the order they are set in the UEFI configuration, starting with the entry you specified (0001 in this case).

🔄 Getting Ubuntu to Boot Automatically from NVME

This above commands will boot Ubuntu fine every time, but this will not "stick" and the next time you boot the board you will face the same error and the OS will not load automatically. So just how do we achieve automatic boots?

Once again, we need to drop into the U-Boot console. This time we can set some UEFI environment variables to store the commands and run them automatically each time the board is booted:

env default -f -a
saveenv
setenv bootcmd_nvme0 "pci enum; nvme scan; efidebug boot add -b 0001 'Ubuntu' nvme 0:1 /EFI/ubuntu/grubriscv64.efi; efidebug boot order 0001; bootefi bootmgr"
setenv bootcmd 'run bootcmd_nvme0'
saveenv

First, we clear any existing variables to ensure a clean slate. Afterwards, we bundle the chain of commands into the bootcmd_nvme0 variable for use later.

Then we point the bootcmd variable to run our custom boot command. The bootcmd variable defines the series of commands that U-Boot will automatically execute during the boot process. Essentially, it's the default action that U-Boot will take unless interrupted or directed otherwise.

Finally, save the variables and you are good to go!

The support for this board and RISC-V is still early for Ubuntu and certain things, such as HDMI output, may not be functional at the time of writing this. I suggest accessing the device through the serial console first, enabling SSH and going from there.

🤔 Conclusion

In summary, installing and running Ubuntu on the StarFive VisionFive 2 board is a journey through the intricacies of U-Boot/UEFI, highlighting the adaptability required when working with emerging architectures.

By manually configuring the UEFI boot process, we can successfully navigate these hurdles and get our favourite distros up and running. This process not only underscores the importance of a solid foundational understanding of system boot mechanisms but also opens the door to the vast potential of RISC-V boards in the world of Linux computing.

Have you ventured into installing operating systems on unconventional architectures like RISC-V? Do you have experiences, insights, or questions about running Ubuntu on the StarFive VisionFive 2 or similar boards? I’d love to hear about your journey and any challenges or triumphs you’ve encountered along the way. Feel free to share in the comments below or reach out to me on Mastodon!

The prospect of running Ubuntu, or the latest Debian versions provided directly by StarFive, hinges critically on having the latest U-Boot and SPL firmware on these boards. However, working through the update process can be a challenge due to fragmented and sometimes confusing documentation.

So, if you are excited about running the latest operating systems on your VisionFive 2 board, then you are in the right place! In this post, I aim to demystify this process, offering a guide on updating your board's firmware through the most reliable methods, whether you're a seasoned tinkerer or a curious hobbyist exploring the potential of RISC-V architecture.

🔍 A Primer on U-Boot and SPL

Feel free to skip over this section if you are just here to update the firmware!

Before diving into the firmware update process, let's take a look at what U-Boot and SPL are and their roles in the VisionFive 2 board.

In the context of the VisionFive 2 board, U-Boot acts as the universal bootloader, a critical piece of software that orchestrates the booting process. It initialises the board's hardware, such as CPU and memory, and sets up the necessary environment for the OS to run. U-Boot is highly versatile, allowing for a variety of boot mediums and providing a rich set of commands for configuration and diagnostics via a serial console, making it an essential tool for developers.

Complementing U-Boot is the Secondary Program Loader (SPL), which is the minimalistic first-stage bootloader that takes action immediately after the board is powered on. The SPL's role is to perform the fundamental hardware initialisations and to prepare the system so that the more comprehensive U-Boot can take over the process.

This two-stage boot process ensures that the VisionFive 2 can handle the complex boot scenarios required by modern computing environments while maintaining the flexibility and robustness needed for the development and deployment of various applications on this RISC-V platform.

🤔 Which Method Should We Use?

We will use the SD Card image method to update the board. While there are other methods to update the firmware, the SD Card image method is preferred due to its reliability and simplicity.

In most cases, if you have not updated the board's firmware, then the latest version of StarFive's Debian release, nor Ubuntu, will even boot in the first place.

You should also use this method if you are trying to use the flashcp command method and hit the u-boot-spl.bin.normal.out won’t fit into /dev/mtd0! error message.

📋 Prerequisites

Hardware you will need:

• A MicroSD card (any size 1GB or more should work)

• A USB flash drive

• USB to UART dongle (optional, but required if you want to run Ubuntu later)

Grab the latest version of the following files from the VisionFive2 Firmware Repo:

• u-boot-spl.bin.normal.out - put this on the USB flash drive

• visionfive2_fw_payload.img - put this on the USB flash drive

• sdcard.img - flash this to the MicroSD card using Etcher

⚒️ Hardware Setup

Set the DIP switches to set the Boot Mode to SDIO

• RGPIO_1 -> 0

• RGPIO_0 -> 1

Install the MicroSD card you flashed earlier, plug the USB drive into any of the USB ports and ensure you have either an Ethernet network connection (for SSH) or HDMI and keyboard and mouse available.

Optionally, if you intend to use Ubuntu later, then you will need to set up the USB to UART device using the GPIO pins. I am using the rather handy Raspberry Pi Debug Probe for this. The leads need to be connected from the board to your UART cable as follows:

• Pin 6 -> GND

• Pin 8 -> RX

• Pin 10 -> TX

🔄 Updating U-Boot and SPL

Power on the board and let it run through the boot process fully. This can take some time, but just let it run until you get a prompt. To run the commands following commands, you can either enter them directly or use SSH.

If you hit the dreaded mipi_0p9: disabling hang during the boot, you will need to SSH into the board as it is likely you are using a 4K monitor, which has been known to cause some issues. The username and password you need are root and starfive.

Next, we need to mount the USB drive so we can access the files. This is required as the image we are using right now is so minimal that it does not include any tools that will let us download the files or even a way to install them.

List the available disks with the lsblk command:

# lsblk
NAME        MAJ:MIN RM   SIZE RO TYPE MOUNTPOINTS
sda           8:0    1   116G  0 disk
|-sda1        8:1    1   200M  0 part
`-sda2        8:2    1 115.8G  0 part

In this case, sda2 is the drive and partition we need. Now we can mount the drive and access the files with:

mkdir usb
mount /dev/sda2 usb

Next, we need to list the MTD devices so we can be sure we are flashing the correct firmware to the correct device:

# cat /proc/mtd
dev:    size   erasesize  name
mtd0: 00040000 00001000 "spl"
mtd1: 00010000 00001000 "uboot-env"
mtd2: 00300000 00001000 "uboot"
mtd3: 00100000 00001000 "data"

In this case, we need to flash the SPL firmware to mtd0 and the U-Boot firmware to mtd2. This is done using the flashcp command and providing the files from the USB drive:

⚠️ Ensure you choose the correct device! ⚠️

cd usb
flashcp -v u-boot-spl.bin.normal.out /dev/mtd0
flashcp -v visionfive2_fw_payload.img /dev/mtd2

You should see output along the following lines if successful:

# cd usb
# flashcp -v u-boot-spl.bin.normal.out /dev/mtd0
Erasing blocks: 36/36 (100%)
Writing data: 143k/143k (100%)
Verifying data: 143k/143k (100%)
# flashcp -v visionfive2_fw_payload.img /dev/mtd2
Erasing blocks: 736/736 (100%)
Writing data: 2943k/2943k (100%)
Verifying data: 2943k/2943k (100%)

Now shut down the board and you are done updating the firmware!

👷 Preparing for Ubuntu

If you want to run Ubuntu, there is an extra step and this is where the USB to UART bridge is required. After flashing the latest firmware, we need to reset the U-Boot environment variables to default values, otherwise, the installer/image will not boot.

To access the U-Boot console, you can use picocom. On macOS this can be installed with brew install picocom.

Next, find the USB to UART bridge device and connect to it. On macOS for example, these devices usually appear as tty.usbmodem devices, so look for it with the following command:

ls /dev | grep tty.usb

Then connect to it:

sudo picocom -b 115200 /dev/tty.usbmodem2112402

You should be greeted with Terminal ready if this was successful.

Next, power on the VisionFive 2 board. You will see a large amount of output, but for a few seconds you will have the chance to interrupt the boot process to allow us to drop into the U-Boot console:

Hit any key to stop autoboot:  2

Now you need to run the following commands:

env default -f -a
env save

The output should look like this:

StarFive # env default -f -a
## Resetting to default environment
StarFive # env save
Saving Environment to SPIFlash... 
Erasing SPI flash...
Writing to SPI flash...
done
OK

Shut down the board again and now you are prepared to run Ubuntu on your StarFive VisionFive 2 board! 🎉

📌 Round Up

Now your StarFive VisionFive 2 board is fully updated and ready to run the latest operating systems! While the process is not very obvious from the official documentation, it is overall fairly simple once you get the hang of it. So, next time you need to update your board, simply follow the same steps as before and you will be up and running in no time.

For the next steps on running Ubuntu, visit the Ubuntu Documentation to find out the different options for getting going with Ubuntu on this board.

Have fun!

Apple recently released macOS 12.3 and, in the process, removed Python 2 from macOS entirely. Python 2 has been deprecated, and EOL, for several years now (Homebrew deleted the formula some time ago), so the removal of it from the base OS is no real surprise.

As always, I would encourage everyone to migrate to Python 3 for better long-term support, but if you have scripts that absolutely must use Python 2, then this can be added back to macOS in a couple of commands.

Adding Python 2 back to macOS

The Python project still hosts the final Python 2 macOS installer package, so we can simply download and install this to restore Python 2 to macOS. It is simple and quick to do this entirely from the command line:

curl -O https://www.python.org/ftp/python/2.7.18/python-2.7.18-macosx10.9.pkg
sudo installer -pkg python-2.7.18-macosx10.9.pkg -target /

Example:

It is as simple as that. Python 2 will now be available again via the python command and will live happily side by side with the system Python 3 installation (which is the python3 command)

This also works in CircleCI, where Python 2 is no longer available from Xcode 13.3 onwards:

Happy Building!

Now that the DST Root CA X3 root certificate has expired, clients that do not trust ISRG Root X1, now the only valid root for Let's Encrypt, will not be able to verify a Let's Encrypt SSL certificate is valid.

For most people, this is not a problem, as the supported platforms list is fairly extensive and covers most modern platforms that end users will be using. In the case of macOS, this is supported from macOS 10.12.1 which was released in October 2016. However, if you use the curl command in macOS, you may find yourself hitting a roadblock, which can be especially problematic in a CI/CD environment.

The Problem 🤔

If you try to curl a website which is using a Let's Encrypt certificate, like Let's Encrypt's own homepage, you will find you receive the following error:

$ curl https://letsencrypt.org

curl: (60) SSL certificate problem: certificate has expired
More details here: https://curl.haxx.se/docs/sslcerts.html

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.

Exited with code exit status 60

Yet if you open Safari and head to the same URL, there will be no problem at all! Why is this?

It appears that the default installation of curl in macOS does not use the default system keychain, or security framework, in macOS. The system keychain includes a copy of the new ISRG Root X1 certificate which allows any apps that use the security framework in macOS to trust Let's Encrypt SSL certificates. Therefore curl does not know this new certificate as it appears that Apple has not updated curl's certificate store to add the ISRG Root X1 certificate - this only works out of the box in Big Sur and higher.

The Solution 🙌

Thankfully all is not lost! We can point curl to include an additional CA bundle to allow validation of Let's Encrypt certificates.

The curl project provides updated certificate stores every few months on their website, so we can use this to resolve the error message seen above.

Firstly, we need to download the certificate bundle. You can either do this by heading to the download page, or by using curl. It is important to note that if you use curl, you will need to use the -k option to skip validation of the SSL certificate as the curl project website also uses the ISRG Root X1 certificate - a chicken and the egg scenario 🐣!

curl -k https://curl.se/ca/cacert.pem -o ~/.cacert.pem

We then simply need to set CURL_CA_BUNDLE to the path of the new CA bundle which contains the ISRG Root X1 certificate. The easiest way to do this is to add it to your .bash_profile:

echo 'export CURL_CA_BUNDLE=~/.cacert.pem' >> ~/.bash_profile

Now reload your shell, or source ~/.bash_profile and you will once again be able to use curl with any URL using Let's Encrypt!

For the vast majority of users, even developers, this is not an issue as who doesn't like extra security - simply click a few buttons instead and you'll be on your way. Yet little thought is given to headless environments, such as those used in continuous integration and delivery! CI/CD is a huge part of development lifecycles these days and with Apple removing useful features silently becoming a common occurrence, it is making setting up CI/CD pipelines harder than ever.

One such instance is the removal of the AllowRemoteAutomation key in Safari 14 and above. In the blog I discuss how I worked around this to provide our customers at CircleCI the ability to keep testing their websites with safaridriver in macOS Big Sur based images.

Testing with safaridriver on Safari 14+ 🖥

It is essential that the AllowRemoteAutomation key is set to true before safaridriver is enabled, otherwise remote control of Safari is not possible.

In Safari 13 and lower, on CircleCI we could typically set up a job as follows:

- run: defaults write com.apple.Safari AllowRemoteAutomation 1
- run: sudo safaridriver --enable

This was all that was needed to set up the environment ready for Selenium tests.

However, try this on a CircleCI image that provides Safari 14 (such as the Xcode 12.5 image based on macOS Big Sur) and you will be treated with the following error message:

SessionNotCreatedError: Could not create a session: You must enable the 'Allow Remote Automation' option in Safari's Develop menu to control Safari via WebDriver.

Turns out Apple silently removed this key, along with IncludeDevelopMenu and IncludeDebugMenu - all of which were very useful in headless environments! There is seemingly no replacement for these either, so you are now forced to enable them with the GUI. This is also true in Safari 15 on macOS Monterey.

Above: How it feels to be a Mac person working in DevOps (🐼=🍎)

AppleScript to the Rescue! 🍎📝

Thankfully all is not lost as we can employ AppleScript to interact with the GUI for us and enable options we need. AppleScript is a powerful tool for automating actions in macOS, although to get it running in a CI/CD environment

tell application "System Events"
    tell application "Safari" to activate
    delay 5

    tell process "Safari"
        set frontmost to true
        click menu item "Preferences…" of menu 1 of menu bar item "Safari" of menu bar 1
        click button "Advanced" of toolbar 1 of window 1
        tell checkbox "Show Develop menu in menu bar" of group 1 of group 1 of window 1
            if value is 0 then click it
        end tell
        click button 1 of window 1
        click menu item "Allow Remote Automation" of menu 1 of menu bar item "Develop" of menu bar 1
    end tell
end tell

(You'll have to forgive the formatting here as Hashnode does not know what AppleScript is!)

This script, while it may not look pretty, achieves the same result as toggling the AllowRemoteAutomation key by working with the GUI in Safari to toggle the same option.

Firstly, we need to open Safari and ensure it is brought to the foreground to prevent any potential hiccups when the rest of the script tries to run.

Next we open Safari's preferences, head over to the "Advanced" section and toggle the "Show Develop menu in menu bar" option. This places Safari in Developer mode, equivalent to toggling the IncludeDevelopMenu flag in Safari 13 and lower.

Finally, the script toggles the "Allow Remote Automation" option in the "Develop" menu.

Phew! Now while this works locally (simply throw it into Script Editor and run it), there are more things to consider when getting it to run within the CircleCI macOS environment...

Getting it to Work on CircleCI 🤔

While running the above AppleScript locally, you may have noticed a permission pop-up occur. If you do not grant this permission, then the AppleScript will not be able to interact with Safari. As the CircleCI environment is headless and non-interactive, this poses a big problem as, surprise 🎉, there is no command line equivalent for accepting/adding permissions.

Working with permissions in the CircleCI macOS environment is a whole blog post by itself, so I will not go into depth here (stay tuned for that post in the future). The short story is that we can manually inject the correct permissions into the permissions database (SQLite-based). This emulates what the permissions GUI does, but is only possible if System Integrity Protection is disabled.

Finally, this needs to be made easy for customers to set up. No one wants to copy and paste a bunch of complex sqlite3 commands and add the AppleScript as a file in their repo. This is where Orbs come in. We can package these commands up into a tidy, single line call to the Orb. By using the osascript -e command we can keep the AppleScript in-line to the Orb as well!

For the full source code, check out the add-safari-permissions command in the macOS Orb.

Summary

Well that's it for this blog post, I hope it was helpful!

While Safari is a small drop in the ocean compared to Chrome, with 10% market share, it is still important that developers are able to test against it with ease in their CircleCI pipelines with a bit of AppleScript magic.

I will be writing more about how we have worked around Apple's weirdness in future posts, so please stay tuned.