Research Log: December 2013

# December 10, 2013

I've been experimenting with manipulating the background color of the code editor to show information. This video demonstrates a crude prototype I built to explore the idea of showing Theseus' code coverage information changing over time:

Some things work well:

It's easy to tell whether the code you're looking at has ever been executed, and whether it's been executed recently.
The bar at the bottom is a convenient way to see the names of recent events. The information density potential is considerably higher than a more traditional timeline, such as the one in Chrome (shown on the right).

But there are also problems:

People tend to see the white areas representing past execution as dissociated white blocks.
When a function is executed, but it contains code that was not executed (for example, a branch of an if statement), I use a lighter shade of gray. I found when I used the same shade of gray as the rest of the unexecuted code, the function looked like it was sliced in two and it was jarring. However, everyone asks why there are two shades of gray.

I made a later prototype where I tweaked the visual style to help with those problems:

The main changes are:

Dark lines connect the events in the list at the bottom to the shaded areas above them.
Old events are clustered together at the far right, leaving more space for the others. The most recent event is given almost half the screen so that it's likely to span most of the width of the code, strengthening its appearance as coverage.

I even took the opportunity to make slick animations with d3:

After talking with Rob, however, I think I'm going to give this idea a rest and try out some other ways of showing code execution. We're currently thinking of developing an icon language that can be used to the right of the code to indicate coverage, amongst other things.

# December 11, 2013

Programming Screencasts

A recurring problem in my research is the inability to get inside a programmer's head while they're programming.

People generally don't think out loud while they're programming. You can ask them to think out loud, but they don't always talk about the aspect of their thought process that you want to hear. You can ask them about their thought process, but that changes what they were thinking. I'm not nearly good enough at experimental design to tease all of those things apart yet.

On the other hand, I know exactly what I'm thinking, right? Well, I've tried analyzing my own thoughts while I program, but it doesn't really work. I can only think about one thing at a time, be it my programming problem or what I'm thinking about the programming problem, which leaves huge gaps.

So I thought of a trick to get around that: record myself programming with a think-aloud protocol then watch the video later. Even if I don't say the "right" things, if I look at the video later I might remember enough of what was going through my head to do some sort of an analysis.

So I recorded myself programming a snake game, and for the heck of it, I uploaded it to YouTube.

Here is part one (21 minutes) in which I make a working game without taking any time to go back and refactor:

In part two (19 minutes), I clean the code up:

I tried watching the videos immediately after recording them, but I found I was still too distracted by the programming problem itself, so I'm giving myself a little more time to forget what I did.

I had a secondary goal of making a video that demonstrates what it's like to solve a problem with code. I love programming, but I know people who hate programming. I suppose this video was an attempt to encapsulate everything I love about it in one place. I think I did all right, considering a comment I received:

Watching and hearing you code was fascinating, btw. … The order in which things happen, for one. The points at which you test. The way you look things up … and work through bugs. … You can tell people those things, but showing is better. The exercise didn't illustrate a programming language, but coding itself.

Exploring a coding screencast by exploring the code

When I showed the video to my advisor, he had a vision of being able to navigate the screencast by pointing at portions of the code. The idea captured me and I whipped up a proof-of-concept last night, which I demonstrate in this video:

Two uses come to mind, one reasonable and one silly:

Education: lecturers often write algorithms in class from scratch, and a tool like this could help students locate and rewatch parts of the lecture video more efficiently.
Documentation: this idea is a little out there, but if people thought aloud about their code as they wrote it, this might be a form of code documentation that really allows you to figure out what the author was thinking when they wrote that abomination.

One major issue is that it's hard to generate a map from code to video timestamps in general. For this prototype, I listened for CodeMirror change events, but I don't like the idea of making a plugin for every editor. I'm wondering if you can't induce the mapping from an editor-agnostic keylogger and the final versions of the files…