On May 25, 2016, around 8am PST, tired and bored, I bought a Super Nintendo console from eBay, with the explicit intent of coming up with some kind of art project for it. When it arrived, I opened the case up, got an idea for something to build, then made absolutely no progress.

In April 2017 I moved across the country with the SNES in tow. Around mid-July, I started tinkering with the SNES again, but I didn’t seriously get back into the project until October. At the beginning of this year, I decided that I would either finish the damn project by May or throw the whole console away.

This is the story of completing the project.

The Idea

The exhibition of console-hacking-disguised-as-art-pieces isn’t new. Probably the most well-known artist in the medium is Cory Arcangel, who exhibited Super Mario Clouds at the Whitney in 2002. More recently, I’ve found Nathan Villicaña-Shaw to be designing really wild stuff, like his Symbiotic SNES project, where you can play four different copies of an SNES game simultaneously with a single controller.

In late 2015, Shared Studios’ Portal Project arrived in San Francisco. It’s a neat idea: you step inside a shipping container, into complete darkness and silence. The only other artifact that’s present is a giant projector screen across from you. Soon, you’re connected with another person who also stepped into a shipping container in some other part of the world. You get fifteen minutes to introduce yourselves and talk about your cities, your interests, your lives. And then you’re kicked out.

I have two older sisters, and, growing up, we used to play on my cousin’s NES at my nene’s house (which I believe we may have even inherited eventually–the NES, not the house). But by the time the SNES came around, my sisters had grown out of video games. In order to play anything multiplayer, I had to be taken to a friend’s house, which, while not unpleasant, was only possible if my mother’s permission and schedule allowed it. Between the four of us, she was the only one who knew how to drive.

My SNES idea was simple: I wanted two Super Nintendos, one acting as Player One and another acting as Player Two to communicate wirelessly with each other. Ideally, one would be set up where I lived and one would be set up in Mexico City or London or Los Angeles, somewhere far enough away to make the experience amazing. My main constraint was that the project had to run on original hardware. No emulators, no tricks. Whatever I built would need to pick up controller inputs and send them somewhere else.

After many months of failed attempts, I finally got the damn thing working, though I did have to scale back the idea to use radio signals instead of wifi, which means that the two consoles are communicating through each other at a maximum distance of about 150 feet. There were two reasons to drop down to radio signals. First, I couldn’t find a wifi enabled device that was small enough or reliable enough to fit inside an SNES console. More importantly, I had no idea what I was doing, and keeping it simpler seemed smarter. To establish a wifi connection, there are a lot of other factors to keep in mind, like setting up a server to handle the communication and ensuring that lag between the devices isn’t problematic. With radio signals, the only major problem would be physical interference, which can be worked around by just putting the two consoles in an open area.

The Setup

The schematics and protocols for how an SNES works have been known for decades. Here’s a write-up from 1996 about the way the SNES processes button pushes from a controller. Although there are seven pins available on an SNES connection, only five pins are used: 5V power, clock, latch, data, and ground. The other two wires do nothing, and probably existed for potential new devices to expand on later on. As well, all of these wires, except data, flow INTO the controller, whereas the data signal is sent back OUT to the SNES console.

Suppose you want to order tacos from your favorite restaurant. They’ve got 12 different kinds of fillings: chorizo, carne asada, pollo, huitlacoche, carnitas, and so on. But you don’t know which fillings are available until you call to place your order. When you call the restaurant, they stop everything that they were doing, and just go down the list of fillings in order to tell you what’s available:

—Hello.
—Hello. We have chorizo, pollo, huitlacoche. That’s it.

In a grossly simplistic format, the console/controller relationship works the same way. You are the console, and the controller is the restaurant. Every ~16.67ms (0.01667s), the SNES console sends out a signal to latch (or store) the state of the buttons. A chip inside the controller keeps track of which buttons are pressed and which aren’t. Then, the controller sends back each button state to the console in 12μs (0.000012s) cycles. The polling sequence of each button is the same: first it’s the B button, then Y, then Select, and so on. This happens 16 times, even though there are only 12 buttons on a regular SNES controller. Again, these additional cycles may have been meant for some kind of controller with additional buttons.

The point of this nerdery isn’t to be pedantic; it’s to point out that the 16-time read cycle happens really fast. In order to implement my idea, I needed a microcontroller intercept the button data when it came from a controller, store it, and then allow the state to pass through to the console. After storing all 16 button states, the microcontroller would need to send all of these to the other console and “replay” them. In other words, someone would need to tap the phone line of the taco conversation, pretend to be a customer, ask for the available taco fillings, and then pass the available fillings back to the customer so they can do whatever they want with the information.

In theory, it’s possible, but in reality the timing was hard to get right. Searching the Net, I found this video which, at least, validated that one could intercept SNES controller data and have it pass back into the console unhindered. I also found projects that created wireless controllers, which gave me some hope, but as far as I knew, no one was sending controller data wirelessly to a different console.

The Analysis (a.k.a. The Many Months)

I don’t know much about hardware hacking. I built a few PCs when I was younger, but that’s like following a recipe. If you stick to the parts list, you’re not likely to screw anything up. My one and only hardware hack was fixing a VCR by replacing its rubber band so that cassette tapes could properly insert and eject.

When I still thought I could do this project over wifi, I bought a Raspberry Pi 3, which comes with an embedded wifi chip. Then I did something very dumb: I opened up the SNES I had and simply soldered five wires to the back of the controller port. I thought that if I loaded the Pi with RetroPie, an emulator that provides support for real controllers, I wouldn’t need to write much original code. Through a forum post, I also found that this idea had merit: at least one other person had soldered wires directly onto the SNES.

I never got this setup working. The Raspberry Pi could intercept the buttons, but it would never send them back into the console, meaning that the game was not playable. I spent months in this sort of limbo of checking and rechecking code and posts and ideas, but I never got it to work. I gave up.

When picked the project back up again, I took a different approach. The SNES is a very valuable piece of hardware because its supply is limited; it’s not a good idea to just solder ports and potentially damage it. I didn’t want to manipulate and destroy any part of the unit until I was certain that I could get something to work. I bought two SNES controller extension cables. I cut one of them open down the middle and stripped off the black tubing. Then, I crimped each of the ten wires (five from each half), which was not fun at all. I didn’t think it was possible for something to be more stressful than soldering, but here I was, spending three hours trying to get a bit of metal to wrap around a few thin strands of copper.

By working with extension cables, I could plug one end into the console and the other end with the exposed wires into the breadboard. With the other spliced half, I plugged its exposed wires into the breadboard, and the other end into a working controller. Hopefully this pictures makes the setup clear:

In the picture:

• Green is 5v power
• Black is ground
• Red is data
• White is latch
• Yellow is clock

Why do this? Breadboards wiring runs horizontally, meaning that a wire plugged into one end of a row is connected to a wire plugged into all the other columns of the row. By splitting the extension cord apart, I could ensure that the controller was still communicating with the console, while at the same time, I could plug separate wires anywhere between them along the row to intercept the signal back into the microcontroller.

Still certain I could do this project with wifi, I bought an Arduino MKR1000, which was smaller than the Pi, and ought to have fit better in the SNES console case. But this didn’t work at all, either. Most of the other SNES controller sketches I found were not using an MKR1000. I discovered that the MKR1000 runs on a newer ARM chip, and older sketches for other boards and chips would not work for it. Ugh.

The Implementation

A problem with a project like this is that I couldn’t quite tell if the software I wrote was the problem, whether the hardware that I manipulated wasn’t working, or whether the idea just had zero functionality at all.

After failing with the MKR1000, I decided to just start all over from the basics. Forget trying to get this to work over any range: I just wanted to see if I could get buttons to intercept reliably.

I found a project which appeared to provide support for intercepting buttons. I bought an Arduino Uno in order to reimplement the project on a known working chip. I kept the breadboard setup, and plugged wires from the Arduino between the wiring for the controller and the console. This, for the first time, worked. I could intercept the SNES controller buttons, and the console played the game at the same time. I was ecstatic.

Except…a lot of the input was being dropped. I would press a button and sometimes the Arduino would register it. I thought that maybe the connections were bad (the crimp job from the spliced extension cord was really bad), but eventually decided that it might’ve just been the software.

The code made use of the standard setup for reading signals on an Arduino. Recall that as far as the relationship between the console and the controller is concerned, the clock and latch wires ought to flow INTO the controller, and the data signal from the controller flows OUT to the SNES.

My setup originally looked something like this:

pinMode(SNES_LATCH, INPUT);
pinMode(SNES_CLOCK, INPUT);
pinMode(CONTROLLER_DATA, INPUT);

...

# we've called the restaurant, check button states on clock cycle
WAIT_FALLING_EDGE(SNES_LATCH);

for (position = 0; position < 16; position++) {

  # within the allotted time, tell me which filling is available
  WAIT_FALLING_EDGE(SNES_CLOCK);

  button[position] = digitalRead(CONTROLLER_DATA);
}

I learned that digitalRead is not time critical. All of the 16 button reads need to happen within 12μs (0.000012s) each. Could this “conversation” between the microcontroller and the controller be too slow?

I came across the concept of direct port manipulation, which, in a nutshell, sacrifices legibility of code for speed. Once I changed the code to read the signals via direct port manipulation, the Arduino could consistently read the data. It took me an incredible amount of time to understand implementing this concept in software. This blog post actually helped me find a project that relied on port manipulation to read SNES controller data, and was exactly what I needed to continue.

Now that I could read signals from a controller, I had to test the next step: could I manipulate controller data going into the console? That is, if I pressed down on the Player Two controller, could I force the Player One character to go down at the same time? The button cycle pattern is always the same. On the fifth cycle, I wanted to send a signal to Port One to force the character to always go down, as if someone were just holding the Down button on the Player One controller. Proving that I could send data to Port One meant that I could deliver any button data I wanted, regardless of where it was coming from (a real controller, a signal generated by software, or from a controller fifty feet away).

I spliced another extension cord open. I plugged one end of it into Port One of the SNES, and I only plugged the data wire into my breadboard. The latch and clock signals came from Port Two. What I wanted to see is whether the timing for controllers one and two occurred at the same time, or at separate intervals. To prove that, I would continue to intercept the latch and clock signals from the console’s Port Two; I would use the Player Two controller to send data; but I would wire the breadboard up so that the data went to the console’s Port One. Whatever button Player Two pressed should also be sent to Port One.

This, to my immense joy, worked. Although I couldn’t find any information on it, it seems that the button latches and clock reads happen simultaneously for Ports One and Two.

The final part of this experiment was to have the two Super Nintendos communicate with each other. Scraping the wifi idea, I bought two NRF24L01 modules. These chips operate on a radio frequency, sort of like the technology behind cordless phones. I wired these up to each Arduino, and wrote a little program to just send a “Hello World” message between the two Arduinos. But I couldn’t reliably send a message from one Arduino to another. Messages were being dropped.

I quickly discovered that while the NRF24L01 operates on 3.3V, and the Arduino can provide 3.3V power, there’s an issue with the regulation. In order to work around this, most guides suggest soldering a capacitor between the positive and negative pins of the NRF24L01. After I did this, the “Hello World” message transmission worked beautifully.

I unplugged everything off the breadboards. It was time to graduate this project from an experiment to a real configuration. I clipped off one end of each of my jumper wires, and soldered the exposed copper directly to the back of each controller port:

On one Super Nintendo, I soldered a wire going from the back of all five pins and plugged them into the Arduino. I then soldered a sixth wire onto the other controller’s data pin. That is, one Super Nintendo had the clock, latch, and data ports for Port One flowing into the Arduino, and it was also wired to send data out to Port Two’s data pin. The other Super Nintendo had the reverse setup, so that one Arduino acted as intercepting Controller One’s signals and sending Controller Two’s signals out. Using the NRF24L01, I had the Arduinos pass each other the button states for their respective controller to the other unit

And finally, everything was working. I could press buttons on Controller One and have it register as an input on the other console.

The Next Steps

All the code for this project can be found on GitHub.

I’d like to take a healthy break this summer before resuming work on my original idea of making two Super Nintendos communicate over wifi. Synchronizing two independent games would be immensely difficult. That is, if one cartridge has a random enemy appear, the other cartridge would need to duplicate that event, so that both players are playing the same game.

But this would not be impossible. There was once a hardware attachment called the XBAND, which fit between the console and the cartridge, and it allowed for Internet gameplay over a modem–pretty advanced stuff for 1995. The most interesting piece of technology here would be how they managed to keep the two games in sync. Unfortunately, the SNES XBAND code has never been made public, so it’s not clear how this was originally done.

In all honesty there might need to be some “artistic license” in providing a functional experience. I can absolutely envision one system being the primary console, and providing all the game and video data to a secondary, “dumber” system, which would be nothing more than an Arduino housed inside of a console, happily receiving and sending real controller inputs, with zero actual cartridge code being executed. I’m still against emulation, but as long as there’s some real hardware being used, what does it matter if it’s only partially an illusion?