Refraktions

Current Version: 2.1.3

Introduction

Refraktions is a generative MIDI sequencer with artificial intelligence. Your taps imprint the AI memory and help it generate loops tailored to your choices over time. Start by tapping the screen in several places. Each tap triggers a sound. These sounds will loop and morph into new compositions over time, evolving indefinitely. More taps will be taken into consideration by AI and steer the app in new directions. The AI memory persists between app launches and the more you use the app, the more it learns about the instruments, pitches, and compositions you prefer, weighing your most recent decisions more heavily.

Conventions of this Manual

When referring to "your device", that can be an iPhone, iPad, or iPod touch with iOS 9.0 or higher (to utilize the CoreMIDI framework). There is not a Refraktions 2.x for Android at this point.

When referring to app settings, settings are indicated like this, and the value they are set to like this.

For example: Set the Background Photo to on.

Table of Contents

  1. Release Notes
  2. System Requirements
  3. App Layout
  4. Customizing the Interface
  5. Using the App
  6. Using MIDI
  7. Controlling Other Synthesizers
  8. Ideas
  9. Contact

Release Notes

2.1.3

2.1.2

2.1.1

2.1

2.0

System Requirements

Refraktions requires iOS 9.0 in order to access all the CoreMIDI functionality used in the application.

Refraktions works best on newer iOS devices with 450 MHz GPU. It will function on older devices but the animations and sound output capabilities will be scaled back in order to insure basic functionality.

High Performance Devices

These devices utilize all the functions and animations.

Medium Performance Devices

These devices have some of the animation and sound output capabities limited.

Low Performance Devices

These devices have most of the animation and sound output capabities limited.

If your device launches in either Medium or Low Performance Mode, you will see that information on the screen when the app launches.

MIDI Functionality

MIDI functionality is fully functional in all modes.

App Layout

Screenshot of Refraktions 2.1 interface
  1. MIDI Indication Light indicates when a MIDI note is being sent or received.
  2. Settings Button opens up the Settings Panel.
  3. Tempo Selector allows the user to change the app's internal tempo.
  4. Sequencer Position Display displays the current measure, beat, and bar of the internal sequencer.
  5. Text Display outputs various messages to the user.
  6. Activity Meters are the small, vertical vars along the right side of the screen. They are a quick snapshot of how busy each instrument is. A dim meter is low activity, a bright meter is high activity.
  7. Jog wheel enables the user to scrub through the composition.
  8. Audio Mixer Overload Light is the set of dim white shapes behind the rotating circles. When the overall audio output of the app begins to peak, these shapes will get brighter.

Customizing the Interface

Settings Panel

Press the Settings Button (gear icon) to open up the Settings Panel. Here you can enable Ableton Link, MIDI Sync, Background Mode, MIDI Mode, set the MIDI Destinations and Instrument Routing, change the musical scale, reset the AI memory, and change the visuals.

Settings Panel

Ableton Link

Toggle Ableton Link for the app. For more information see the Ableton Link website.

Ableton Link screenshot

The default is off.

MIDI Sync

Select whether the app synchronizes to an external MIDI clock, sends a MIDI clock, or neither.

Sync to External MIDI Clock

Refraktions will listen for an external MIDI timing clock coming from an external sequencer or DAW. It will sync to the incoming tempo and its internal sequencer will respond to incoming MIDI start, position, and stop messages.

Send MIDI Clock

Refraktions outputs a MIDI timing clock at the tempo of its internal sequencer (set by the Tempo Selector). It also sends MIDI start, stop, and position messages when the sequencer is scrubbed or reset.

None

Refraktions ignores any incoming MIDI timing clocks and does not output a MIDI timing clock.

MIDI Clock Settings screenshot

The default for MIDI Clock is None.

Background Mode

Background Mode enables the app to continue to play music or send MIDI notes after it has been moved to the background (by pressing the Home button on your device).

Background Mode screenshot

The default is off.

MIDI Mode

MIDI Mode turns the app and its AI into a powerful MIDI sequencer capable of controlling up to 7 synthesizers at once. See the Controlling Other Synths for step-by-step instructions on how to get started.

MIDI Mode screenshot

Note: The app's internal sounds are muted in MIDI Mode to enahnce performance and remove audio distraction.

The default is off.

MIDI Destinations

The MIDI Destinations panel allows you to select one or multiple destinations to receive MIDI notes from Refraktions' sequencer. This includes other MIDI-enabled iOS apps

MIDI Destinations screenshot

Instrument Routing

Instrument Zones

Refraktions has 7 horizontal instrument zones, stacked top to bottom. Each corresponds to a different synth sound in the app's internal sound bank. Each zone has its own note sequences.

Diagram of instrument hit zones.

These 7 zones output MIDI notes when MIDI Mode is on.

The default settings are:

Changing instrument routing

In the Settings Panel you can change each instrument zone to output to any MIDI channel (1-16). This allows you to control up to 7 external synthesizers at once, or multiple zones can be output to a single MIDI channel, creating a more complex MIDI sequence.

Instrument Routing screenshot

Musical Scale

Choose any of the following musical scales.

List of all the musical scales.

The default is Major.

Reset AI Memory

The app's artificial intelligence has a memory that records the user's selections of instruments and keys. Over time, the memory gets imprinted more deeply and when generating new compositions, it chooses instruments and pitches the user has favored in the past. If you'd like to start from scratch and wipe the slate clean, tap this button in the settings panel.

Reset AI Memory screenshot

Visuals

Background Photo

Toggle the Background Photo on or off.

When it is off, the background is mostly black.

The default is off.

Information Display

Toggle the Information Display on or off.

This is the visual linework that includes the MIDI Indicator Light, the Tempo Selector, the Sequencer Position Display, the Message Display, the Activity Meters, and the Audio Mixer Overload Light.

The default is on.

Visual Settings screenshot

Using the App

Setting Information Display to on will make it easier to observe the following functions, but they will work whether Information Display is on or off.

Inputting notes

This is the core functionality of version 1.0. Simply tap or sweep your fingers on the screen and you will hear different synthesizers play.

The sounds will loop at different rates until they slowly fade out and disappear. When they disappear, the overall composition is assessed by the AI and new sequences are generated. The result is that each time the app is used it outputs a unique composition that morphs and evolves over time. The more you use the up, the more the AI learns about what instruments and keys you prefer and creates new compositions accordingly.

Changing the Tempo

To get the Tempo Selector to appear, drag two fingers up or down the screen.

Screenshot of Tempo Selector.

The selection bar helps you can specify a new tempo.

You can touch and drag multiple times to access tempo ranges that are initially off-screen.

This also changes the outgoing MIDI timing clock if MIDI Clock is set to Send MIDI Clock.

When the app is first installed, the default tempo is 120 bpm. Once you change the tempo, that will be stored and reloaded the next time the app launches.

Scrubbing the sequencer

Emulating the jog-wheel on classic hardware MIDI sequencers, you can change the position of the playhead by spinning the outside ring of the sequencer circles.

Screenshot of the sequencer scrub gesture.

When the app is in MIDI Mode, this action sends the corresponding Song Position data via MIDI.

Resetting the sequencer

To reset the sequencer, draw a counter-clockwise circle on the screen. This is an easy way to clear out a busy song and start from scratch.

Screenshot of the counter-clockwise gesture.

This removes all the sounds and animations, stop all MIDI notes, and rewinds the internal sequencer to the beginning of the loop. If MIDI Clock is set to Send MIDI Clock, then a MIDI Song Position of 0 0 is sent.

Note: This gesture does not move the internal sequencer back to 0 if MIDI Clock is set to Sync to External MIDI Clock because the position is being controlled by an external device. However the gesture still removes all the sounds and animations.

Resetting an instrument

To reset an individual instrument instead of all of them, swipe right in its zone and all of its notes will be removed from the current composition.

Screenshot of a right-swipe gesture in an instrument zone.

Setting Up MIDI

MIDI Mode

Note On, Note Off

When MIDI Mode is on, Refraktions translates screen taps into outgoing MIDI notes.

At the beginning of the screen touch, a MIDI "note on" message is sent.

At the end of the touch, a MIDI "note off" message is sent.

Note Key

The key of the MIDI note is determined the horizontal location of the tap.

The left side is low, the right side is high.

Diagram of key zones.

Note Channel

The note's MIDI channel is determined by the vertical position of the tap.

The top is MIDI channel 1, incrementing down to channel 7 on the bottom.

Diagram of MIDI channel zones.

You can change the positions and channels in the Settings Panel.

Sync to External MIDI Clock

Standby

When MIDI Clock is set to Sync to External MIDI Clock, you'll notice the Refraktions sequencer is not moving. The numbers in the Sequencer Position Display aren't updating and the circles aren't moving. This is because the app is waiting for a MIDI start or position message from an external sequencer.

Start, Position Messages

Once a MIDI start or position message is received, the Refraktions sequencer will play at the tempo of the incoming MIDI timing clock.

Stop Message

The sequencer will stop when a MIDI stop message is received.

Send MIDI Clock

When MIDI Clock is set to Send MIDI Clock, Refraktions outputs a MIDI timing clock. Other devices can listen for this clock on "Network Session 1" (as set up in Controlling Other Synthesizers). In this scenario, the Refraktions internal sequencer is the master and the external devices are the slaves.

Changing the Tempo

You can change the tempo of the MIDI timing clock with the Tempo Selector.

Restart from Position 0

If you perform a sequencer reset gesture, the app outputs the following messages:

This is useful for getting the external devices to sync up to the same position as the Refraktions sequencer.

Enable Ableton Link

Enable Ableton Link in the Settings Panel.

Changing the Tempo

You can change the tempo of the Link Session with the Tempo Selector. Incoming tempo requests will be displayed on the Text Display.

For more information, see the Ableton Link website.

Controlling Other Synthesizers

Refraktions can send MIDI information to external synthesizers, other iOS apps, or a DAW on a computer.

Option 1: Control external hardware synthesizers

Refraktions --> MIDI connector --> External hardware synthesizers

  1. Connect your device to your synth using an adapter like iConnect*.
  2. In the Settings Panel, set MIDI Mode to on.
  3. In MIDI Destinations, selecte iCONNECT (or whichever adapter you're using).
  4. Select your instrument routing.
  5. Start tapping the Refraktions app. Your external synthesizer is now being controlled by the Refraktions internal sequencer and AI.

*Alternately you can connect to your DAW and use it to route the MIDI to your external synthesizers.

Option 2: Control iOS synthesizer apps

Refraktions --> Other iOS apps

  1. Launch the target apps and enable their MIDI. Please consult each app's documentation for specifics.
  2. Go to the Settings Panel and turn MIDI Mode on.
  3. Next, go to MIDI Destinations and select your target apps.
  4. Select your instrument routing.
  5. If you'd like Refraktions to send MIDI while in the background, then turn Background Mode on.

Options 3: Control DAW synthesizers

Refraktions --> Network Session 1 --> DAW --> Software Synthesizers

1. Set up Refraktions MIDI

Go to the Settings Panel and turn MIDI Mode on.

2. Connect your phone to your computer

This simplest way to do this is plug your device into your computer via a USB cable.

Alternatively, you can have your device and computer join the same WiFi network.

3. Open Audio MIDI Setup

Audio MIDI Setup application icon.

Audio MIDI Setup is an application on the Mac located in Applications > Utilities.

Find it and open it.


4. Open the Network Window

Locate the Network icon (you may have to scroll around if you have a lot of MIDI devices) and double-click it.

Audio Midi Studio screenshot.

5. Create a Network Session

This opens the MIDI Network Setup window where you can create a session, find your device, and monitor the connection. This is the window you'll spend most of your time when establishing a MIDI connection.

Screenshot of inactive Network window.

If there are no sessions in My Sessions, click on the + button to add a new session.

The default session name is "Session 1."

If Session 1 is already in My Sessions, make sure that its checkbox is clicked.

Also make sure these settings are set:

The window should now look like this:

Screenshot of active Network window.

6. Open Refraktions

Go to your device and launch Refraktions.

Go to your computer and you'll see your device's name appear in Directory.

Screenshot of Network window with device in directory.

7. Connect

Highlight your device's name and press the connect button.

If the connection is successful and MIDI Mode is on, you will see this on Refraktions:

Screenshot of Refraktions when a MIDI connection is established.

In the MIDI Network Setup window, you will see this:

Screenshot of Network window with active connection.

8. Set MIDI Destination

Return to Refrkations and open the Settings Panel.

Go to MIDI Destinations and make sure Network Session 1 is selected.

9. Routing

In the Settings Panel, select your instrument routing.

10. Set up software synths

In your DAW, set up 7 software synths. Set each synth's MIDI In to Network Session 1 and give each synth its own MIDI channel.

Tapping the Refraktions app will now be playing the synths in your DAW.

Troubleshooting

If you have trouble creating a connection, try the following:

Ideas

Summing MIDI Channels

In any of these examples you may find it useful to sum your MIDI channels.

For example, if you wanted the top part of the Refraktions screen to play an external synthesizer, and the bottom part of the screen to play another, you could set Instrument Zones 1, 2, and 3 to output on MIDI channel 1, and Instrument Zones 4, 5, 6, 7 to output on channel 2 (customizable in the Settings Panel).

Alternatively, summing can be achieved in some DAW's or stand alone applications like MIDI Patchbay.

Real World Usage

On a personal note, I created Refraktions to be a tool to use in my own live performances and recording sessions. One of my main challenges with loop based music is finding the balance between the entranced state that rhymtic music can evoke and a composition becoming too repetitive and grating. Refraktions become my solution for that.

Live Performances

In live performances, I use Refraktions in MIDI Mode to control several of my hardware synths, including a modular set up. Refraktions takes care of the notes and sequences, evolving organically over the course of the performance. If I need to add more notes, I tap the screen. If I need to clear everything out, I reset the sequencer. But for the most part I leave it alone. I trust it to make its own composition while I'm freed up to do the fun part - turning knobs on the synths and effects pedals.

Each show is unique, a collaborative performance between myself and the AI inside the app. And each time I use it, it creates compositions that are my style. I tend to tap the lower area of the screen, which are the lower-pitched synths, accented with the faster percussion hits of instrument zone 1. The AI has learned my personal style, and generates new notes that favor my preferences.

Studio Recording

In the studio, I've set up Refraktions to output MIDI sequencers into a DAW MIDI channel. I record the incoming notes for several minutes, alternately inputting notes manually and waiting to see how Refraktions morphs those sequences.

Once complete, I audition different synth sounds and samples with newly recorded MIDI notes, deleting the lackluster sections and saving the intriguing ones. It's become a quick way to generate melody and rhythmic sequences to use in a song.

Contact