Lyle Klyne » Interaction & User Experience Designer – Relink


The freedom of hand written notes combined with the power of digital organization. A conceptual smartpen and companion app.

Relink hero image


Textbooks haven’t adapted to the current landscape of digital learning. My team mate and I worked with an industry sponsor to imagine the future of college textbooks. Through research, we found that students have a difficult time taking notes with digital and physical textbooks and sought to improve this experience.

We conceptualized a physical textbook with modified printing. When paired with a smart pen, this textbook would allow analog note taking that resulted in digital notes. We developed a gestural language for the pen as well as a companion app to view notes.

Relink being used on an ipad

##1st, Discover

####Subject Matter Expert Interview: Karl Bakeman There are a lot of players and competing voices in the educational system. One would think students would be prioritized from a content and publishing perspective, but this is not the case. For the most part publishers target teachers and in some cases administrators and deans — students, the end users, are largely forgotten.

This is due to corporate consolidation of the educational publishing industry and the fact that most organizations are publicly traded and owned by private equity firms. Because the people paying for a product (students) are not the ones making the decision to purchase the product (teachers / administrators), the educational publishing industry is very similar to the pharmaceutical industry. Considering students are the ones paying for this content, it’s a shame that they are commoditized and their interests, receiving a quality and engaging education, are not the top priority.

####Survey We were leaning towards improving the note taking experience and decided to conduct a survey to validate our idea that problems actually existed. We surveyed fourteen college students and graduates and asked how frequently they take notes in notebooks, on laptops, tablets, and in textbooks. Out of the fourteen people we surveyed, all of them took notes by hand in some fashion. Comments suggested that people enjoyed this experience, but despite this fact, everyone said they took notes on a laptop.

Relink survey results visualized. Takeaway: everyone takes notes on laptops but lots of people still take notes by hand.

Despite the fact that people actively took notes by hand and enjoyed the experience, many expressed frustration that these notes were harder to maintain and organize than notes taken on a computer.

“I’ve adapted to taking notes on my laptop because it’s easier to transfer them into a report later. ”

Participants mentioned that hand written notes allowed were easier and more flexible, but through secondary research we also found that they help with retention of information.

“Students who write out their notes on paper actually learn more.”

— Mueller and Oppenheimer

Competative analysis

We had identified a key problem: people like to take notes by hand but make compromises to do so. The next step in our design process was to examine the current playing field.

Note taking apps


Restrictive: Current solutions don’t fully embrace handwriting. The user must learn new habits and write on a screen.

Complicated: Note taking apps such as notability provide plenty of features but are overly complicated. Simple tasks like highlighting require wading through several options for customization and then highlights don’t align with the text.

Livescribe is a smart pen that comes closest to emulating the hand writing experience. The way it works is a pen uses a camera to read an invisible grid of dots on a screen to determine where a user is writing. These notes are then digitized. It is quite a cool system, yet is not a perfect fit for the academic experience.

  • Livescribe focuses mainly on interaction with a blank slate, not textbooks or other printed material.

  • Livescribe also does not fully embrace the potential of a classroom-based learning environment by fostering interactivity.

Research Findings

  • Students make trade offs when taking notes. When accessing digital course material, referencing notes directly to the text is hard.
  • This is even harder with a physical textbook and digital notes.
  • Music annotation was laborious with current software solutions, and note taking in general was lacking on ebooks.
* * *

2nd, Design

We began to design our own pen that would fit perfectly in the classroom and reading environment. The first step was to storyboard out hero scenario.

Relink Storyboard

This breaks down into four specific tasks.

  1. Creation of artifacts (notes, diagrams) within the text.
  2. Marking and highlighting information within the text.
  3. Interacting with new media types within the text.
  4. Reviewing, organizing, and searching captured information.

Relink Scenarios

Feedback was identified as a crucial aspect of the design. In addition to providing feedback via a nearby device, we opted to include an LED on the tip of the pen to reinsure the user that their notes and actions were being recorded.

Relink Scenarios

Gestural Language

We developed our gesture set through user studies. The study included four college students or recent graduates. The first step involved asking participants to take notes as they usually would with a pen. For the second step, we asked participants to imagine they had a smart pen and perform certain actions. We consolidated similarities in the gestures into a unified gesture set.

  1. Highlight: underline
  2. Highlight large block: brackets
  3. Highlight important: star
  4. Tag: hashtag + tag
  5. Definte: touch + hold
  6. Interact: touch icon

Relink Scenarios


For the companion app, we wanted information to be viewable in two views: overlayed on the textbook as it was captured, as well as in a consolidated view. When shifting between these two views, animation helps the user keep track of how their notes are consolidated.

* * *



We presented Relink to Karl Bakeman of A. A. Norton. We were asked why there had to be a physical pen to accomplish this task instead of using a smartphone camera. Although a product relying on existing devices could be more cost effective and accessible, we felt that the direct feedback provided by the pen would be enough benefit to warrant a physical product. Also, based on our research of existing products we assume that this device could be manufactured on a large scale for relatively low cost.

We had also pitched this product as a financial benefit for A. A. Norton. Our assumption was that since people would be marking up their textbooks, they would become a more personalized item the student would want to hold on to. This would provide the additional benefit of cutting down on the resale market, one of the biggest hurdles for textbook publishers revenue. Karl took a different approach. He said that if the textbooks were more like paperback workbooks, students wouldn’t feel bad about marking them up and the resale market would still be eliminated.

Next Steps

Due to time constraints, we spent the bulk of our time in concept development and focusing on the initial gestures. It would be interesting to further test and prototype new applications for this device. These could include interactive surveys, new approaches to composing music in a textbook, or access to interactive media.


Data Exploration

web app supporting exploration and comparison