Here’s the code I discuss in this post, the README provides a deeper technical discussion.

A little while ago I set myself a challenge to see how I could use a potentiometer and a button to build a novel user interface. It was a kind of discovery / study process, where I wanted the end result to evolve naturally through exploration. To begin with, I had set out to just use a Raspberry Pi along with these two components. But, the ability to establish “value” for the potentiometer would be complicated through the GPIO and software alone, so a special system, called an Analog-to-digital converter (ADC), is necessary. These are sometimes built into microcontrollers, and even though I also have an ADC handy, I wanted to try out one of the microcontrollers I have.

Microcontrollers (MCU) aren’t the easiest thing to work with and their cross-platform, free, open developer tools are kind of lagging in quality relative to those available for general software development. There’s plenty of reasoning for this, but it did set me outside my comfort zone. The MCUs I have: an older MSP430, an Arduino, and an Espruino Original, made it challenging for me to get a consistent potentiometer reading when I wasn’t using the Arduino / Energia system, which I also found to be clunky and confusing at times. The Espruino would’ve been great as I’m much more familiar with JavaScript, but due to an issue with the USB connector, it became too much effort to keep the device connected to my computer.

Eventually I got my hands on the new Raspberry Pi Pico, an ARM-based microcontroller that can use MicroPython. In only a few lines of code in MicroPython I was getting a reading from 2 potentiometers on the Pico. Although using the Thonny IDE is a little painful, similar to the Energia / Arduino IDE, I managed to get a hang of it.

It was during this process I had also started getting into Puredata so that I could potentially have a way to explore uses for whatever interface I was building. During my research, I found a really nice project called rePatcher, that was basically an Arduino controller for Puredata that worked over serial. I’ve never used serial before, but I’ve always wanted to learn, so I dug into the code to figure out how it all worked. Interestingly, both the Arduino code and Puredata patch were relatively straight forward, nothing spectacular or clever, just buffer writing and a handful of readily available utilities. I resolved to get my Pico working similarly to the project, with only 1 or 2 components rather than the full interface they had designed.

It was fairly easy to get a single potentiometer working. I managed to get the ADC reading as a single byte, and send that to Puredata via the comport patch. An additional potentiometer wasn’t trivial, but I got it working without headache. At this point I was having a bit of trouble with Puredata, as it requires a bit of additional effort to acquire and use patches, using the correct audio drivers (I have a Linux system, so I need to use JACK, and have all the correct input, output, and buffer settings), and then understand how everything connects together. I remembered that the browser has an API called “Web Serial” and due to my previous experience with Web Audio, I figured I could explore this as a faster alternative to Puredata for prototyping some ideas. Due to the vast amount of existing code for JavaScript, along with my experience, and the ease of running and sharing web-based projects I expected this to be much easier to get something running.

As it turns out, it is very easy to get up and running with web serial.

As its still a fairly new and somewhat controversial API, there isn’t a huge amount of tutorial or example code to found on the Internet, but what is available makes it simple to get started. Most of the process I went through to get a nice interactive synthesizer working has been documented in the project’s README and code comments. From here, I feel inspired to focus on creating more interfaces with the Pico, and learning more about how I can get it communicate with the web browser, and other platforms as well. Web serial appears to be a comfortable tool to do doing rapid prototyping, but I think there’s a lot more to learn, from a hardware, firmware, and software point of view. Here are some areas I’d like to explore:

  • How to build interface circuits in a correct way, such that I can find a modular and well-engineered approach to designing and creating them.

  • Writing effective code for the Pico, in both MicroPython and C, also so that it can be put together in a modular way for interface variations.

  • Structuring and parsing serial data correctly, and utilising the serial interface in the correct way for building controllable web and native apps.

  • Gain a better understanding of streams, in JavaScript, MicroPython, and C, such that I can more effectively handle data. Dual core programming on the Pico might also be part of this.

  • Build abstractions for the interfaces to make it easier to integrate them with music-making apps, games, and 3D modelling tools.

  • Loading code onto the MCU directly from the web browser, rather than via an external IDE.

I plan to incorporate this piece of learning into the challenge that I created for myself. It feels as though this is a big hurdle out of the way, so it should be straight forward to finish. Being able to use the Pico makes it much easier for other people to duplicate this project, plus the browser means it’s easy to build all kinds of apps to use with the project as well, and distribute it on the Internet for free.