Elder Scrolls III: Morrowind
Automated Game Play
Colin Fahey

1. Introduction

I never played a fantasy Role-Playing Game (RPG) on the PC until a friend of mine, Greg Moulton, introduced me to "The Elder Scrolls III: Morrowind".  This video game was very popular in the middle of 2002.  It is an amazing game.  I enjoyed playing the game despite my general lack of interest in the fantasy genre. 
In this video game, a character has amounts of skill in various kinds of activities.  As in real life, a character can increase skill in an activity by practicing the activity.  For example, a character can repeatedly cast magic spells to improve the ability to cast magic spells.  The character can simply stand in an isolated place and cast magic spells repeatedly and thus improve at casting magic spells.  Also, the character can simply stand in an isolated place and jump repeatedly, or run in circles, to improve the character's athletic ability. 
Tedious practicing can be made easier for the player by using computer automation.  Thus, a player can "practice" activities without the need to be near the computer!  My friend Sam Robertson and I were taking turns playing the game one evening, and he inspired me to spontaneously assemble electronic parts and software to automate game playing.  After approximately one hour, we were watching the game being played by the computer itself. 
Morrowind : Next to a bed, ready to cast a spell.
Improving the character's ability to cast magic spells in a specific category of magic spells, such as the "Destruction" category of magic spells, can be achieved by repeatedly casting any specific magic spell within the specific category of magic spells. 
Specifically, we need to press the "left mouse button" to cast the magic spell.  Every few seconds we can cast the magic spell again.  Eventually we will run out of "Magicka", a kind of personal energy, and we need to rest before casting more magic spells. 
To rest, we need to press the "space key" while we are in the proximity of a bed, thereby "using" the bed.  We also need to dismiss a message box that appears and specifies the duration of the rest period. 
We can automate the entire process. 
A player could use the in-game "console" to cheat in a more direct fashion.  For example, a player can simply type in a command to request the creation of any material object or to acquire any personal attribute or amount of skill.  However, my goal was to be more "honest" in my cheating!  Supplying keyboard and mouse input to play the game seems more honest than using a mechanism outside the context of game play (i.e., the console) to manipulate the game status. 
A second computer, with Visual C++ and my program
I wrote a simple program to issue commands to the serial port.  These commands engage and disengage relays (essentially "buttons") to control the input to the computer running the Morrowind video game.  The program has time delays that correspond to typical time delays of a human at the mouse and keyboard.  Also, the program has different logic to deal with the different kinds of tasks (spell casting, resting) that it must trigger in the other PC. 
Relay board controlled by serial (RS-232) port
This board accepts commands over a serial line (RS-232).  The communication parameters are: 9600 baud, 8 data bits, 1 stop bit, no parity.  Commands can control the 8 relays on the board.  A command can turn any relay on or off. 
Left mouse button, connected to a relay via clips
I opened up the mouse and attached alligator clips to the terminals of the left mouse button on the tiny circuit board inside the mouse. 
Thus, the relay board can control the state of the left mouse button. 
Opened keyboard, keys removed, sheet with traces
When you open a modern PC keyboard, it is just three sheets of paper-thin, flexible plastic.  It's a sandwich, with the top and bottom layers covered by metallic paths that represent the organization of the key buttons.  The middle plastic sheet has holes where the key buttons are located, but otherwise insulates the top sheet from the bottom sheet. 
Anyhow, it's easy to locate the space bar button paths. 
Close-up of connection to space bar circuit paths.
Attaching alligator clips to the pair of paths that, when combined, lead to the "space bar" keyboard message, was easy. 
Thus, the relay board can essentially "press" the space bar on command. 
Morrowind : Next to a bed, ready to cast a spell. (view #2)
We can press the left mouse button to cast spells again and again.  With a really "cheap" spell (low "Magicka" cost), we can cast at least 30 times before we need to rest.  This fact is essentially built-in to our simple C application on the other PC. 
Morrowind : "Using the bed", rest duration dialog box.
After casting a spell 30 times, the C program assumes we need to rest, and it assumes we are close to a bed.  Therefore, the C program issues a serial port command that triggers a relay on the relay board that essentially "presses" the space bar key on the keyboard attached to the Morrowind video game computer. 
Thus, we "use" the bed, and this dialog box appears, asking how long we wish to rest.  The C program assumes this dialog appears on the screen, and also assumes that the mouse cursor lies on the appropriate button within the dialog box (which is a condition that must be set up, just once, in advance), and proceeds to trigger a left mouse button to select the desired option. 
The resting starts and automatically completes, and we are ready to cast the spell 30 more times! 
Every 100 casts (or so it seems), a message appears on the screen indicating that our skill in this area has improved by 1% (essentially). 
Morrowind : Sometimes we advance in level!
If we improve any skill by 10%, we advance to a new "level" of accomplishment in the game.  So, since each 1% of skill takes about 100 practice casts, we advance levels every 1000 casts, and we have to rest about 30 times in the process!  But, thanks to automation, we can improve our character while we sleep...literally! 
Me and my friend Sam Robertson discussed how to handle the "level advancement" dialog box.  We need a bit more mouse control to automate the selection of character attribute boosts.  It can be done, but it's tricky because it involves "moving" the mouse cursor around via automation. 
contact information