Advanced trial order options in OpenSesame

Thu 24 April 2014 by Eoin Travers

At present, open source experiment-builder OpenSesame allows for a block of trials to be run in one of two orders, sequential, and random (with the added option of skipping a given number of trials from the start of a block, running the skipped trials at the end, and conditionally cutting a block short).

In a recent experiment, I was faced with a more complicated design, which I'll outline briefly.

Randomising with constraints

In my experiment, participants were presented with 3 versions (A, B, and C) of 9 problems (1-9). They were also presented with an additional 27 filler trials, for 54 trials in total. While presenting each participant with 3 different versions of the same problem wasn't really an issue (I hope), we didn't want any participants to end up solving multiple versions of the same problem in a row. To this end, we decided on running trials in a random order, with the constraint that no two versions of the same problem occured within 5 trials of one another.

Generating trial order

At the start of the experiment, stimuli were defined as a list of Python dict objects, as follows (yes, the stimuli were all pictures of animals):

real_trials = [\
    {'probe': 'carrots', 'correct': 'bamboo', 'foil': 'rabbits', 'strength': 'strong', 'property':'x'},\
    {'probe': 'carrots', 'correct': 'bamboo', 'foil': 'horses', 'strength': 'moderate', 'property':'x'},\
    {'probe': 'carrots', 'correct': 'bamboo', 'foil': 'foxes', 'strength': 'weak', 'property':'x'},\
    # ...
    # 27 lines in total
    # ...
    {'probe': 'foo', 'correct': 'bar', 'foil': 'foo', 'strength': 'bar', 'property':'foo'}]

filler_trials = [\
    {'probe': 'bar', 'correct': 'foo', 'foil': 'bar', 'strength': 'bar', 'property':'bar'},\
    # ...
    # 27 lines like this
    # ...
    {'probe': 'foo', 'correct': 'bar', 'foil': 'foo', 'strength': 'bar', 'property':'foo'}]

I don't much care about the filler_trials, so long as none of items 0-2, or 3-5, (etc.) from real_trials are shown within 5 trials of each other.

Next, I used a little bit of Python wizardry to generate a random list of stimuli numbers from 0-26, interspaced with blank None entries, where the filler trials will go, check if they satisfy my 5 trial criterion, and if not, keep generating random orders until they do.

# Functions needed
from random import shuffle

def floored(val, floor):
    # Returns the remainer of val / floor, unless val == None,
    # in the case of the filler trials, which can be left as they are.
    try:
        return val % floor
    except TypeError:
        return None

def check_proximity(data_list, distance, floor):
    # For every entry in the order, apart from the last 5...
    for i in range(len(data_list)-distance):
        # Figure out which problem the current trial belongs to.
        home = data_list[i]
        home_floor = floored(home, floor)
        if home != None:
            # If the current trial isn't a filler
            for d in range(1,distance+1):
                # Check if any of the next `distance` trials belong to
                # the same problem
                target = data_list[i+d]
                target_floor = floored(target, floor)
                if target != None:
                    if target_floor == home_floor:
                        # Found two trials from the same problem too close
                        # together.
                        return False
    # Made it through ok.
    return True

order = range(27) + [None]*27 # Use None for filler trials for now
shuffle(order)

min_prox = 5
while check_proximity(order, min_prox, 9) == False:
    shuffle(order)
    # Shuffle until it passes the test
print 'Order set!'

# Substitute in the filler trials
filler_order = range(27,54)
shuffle(filler_order)
iter_filler_order = chain(filler_order)
for i in range(len(order)):
    if order[i] == None:
        order[i] = iter_filler_order.next()

After a few iterations (depending on how close you allow your repeats to be), order will be set appropriately, and you can continue.

order = [8, 25, 20, None, None, None, None, None, 21, 7, None, 24, None, None, 5, None, None, 15, 2, None, 3, 16, 14, 9, 17, None, None, 13, None, 11, None, 19, None, None, None, None, None, 10, 4, 0, None, 12, 6, None, 23, None, 26, 1, None, 22, None, None, None, 18]

Getting OpenSesame to play ball

While all this produces an appropriate list of numbers with which to select your trials, it doesn't solve the problem of getting OpenSesame to run trials in this order.

To do this, we have to circumvent the software's normal way of doing things a little. Usually, in OpenSesame, we set trial stimuli in a Loop object, entering information into a handy table. Crucially, what goes into this table can also be the name of an OpenSesame variable (enclosed in square brackets), as shown below.

Setting the Loop object with OpenSesame variables

This means that we can set our own trial order from the Python script by setting the values of these OpenSesame variables, and then having the experiment run the sequentially.

This happens as follows.

# Set trial order as OS Variables

# Join our lists of stimuli information
all_trials = real_trials + filler_trials
for loc, stim in zip(range(54), order):
    a_trial = all_trials[stim] # Select data for this trial.
    exp.set('probe_%i' % loc, a_trial['probe']) # `loc` being a number from 0 to 54
    exp.set('correct_%i' % loc, a_trial['correct'])
    exp.set('foil_%i' % loc, a_trial['foil'])
    exp.set('strength_%i' % loc, a_trial['strength'])
    exp.set('property_%i' % loc, a_trial['property'])
    exp.set('stimuli_number_%i' % loc, stim)

And that's it! Maybe this will come in useful at some stage.