Making It Stick — A Developers’ Implementation Plan

How to use flashcards and coding to really retain concepts

During a holiday around Europe with plenty of downtime between destinations, I started churning through the e-book ‘Make it Stick’ by by Peter C. Brown,‎ Henry L. Roediger III and‎ Mark A. McDaniel. As a Java developer who feels they have a lot of catching up to do in the industry, I thought I needed better strategies for learning than the ones I have been taking.

A small disclaimer to the link above, it’s an affiliate link and I get a small bonus for sales — but it won’t cost you any more. I do truly believe in the book though, which I’ll hope you’ll see from the article:

With previous textbook readings I’ve done, I didn’t feel I was retaining the information I needed. More specifically, I was getting overall concepts from books but it only felt like I retained a tiny percentage of what I needed.

I’m a big believer in ‘learning by doing’, rather than just passively reading text, and this is borne out by several sources, including ‘Make it Stick’1 and the much lauded ‘Learning how to Learn’ MOOC. 2

Developing ‘experience’ yourself is what matters, not how many times you’ve read the text.

The problem was that the book itself didn’t give extensive implementation plans — quite intentionally as it didn’t want to be prescriptive. But where does that leave us struggling lifelong students of Software development?

I needed a way of proving whether this system could work before investing too much time in it, and I think I found a system that worked.

Simply put, to get the maximum value out of this you need to:

• Identify subjects worthy of investment
• Schedule time to practice and learn
• Generate real experience
• Maximise your RoI

Identify subjects worthy of investment

Its important to decide which subjects warrant this investment because that’s exactly what it is. You will definitely get through material slower if you take this approach but you will improve your recall, as I found.

During the experiment I made a decision to concentrate on learning some of the Core Java trails because I felt that you will always build on top of these fundamentals, and can re-use them in a variety of contexts.

Schedule time to practice and learn

In order to actually get set up, you will need a way to test yourself. ‘Retrieval practice’, whereby you actively look to recall what you’ve learned rather than simply reading has been proven to be more effective than other methods for recall3. Flashcards are one such approach.

I used the website www.cram.com to build up a collection of flashcards which I added to every time I came across a concept that I felt would be important to recall. I preferred this to paper flashcards because I could sync across my tablet, PC and phone, and have a set of tests that were ‘good to go’ whenever I was on public transport, a lunch break, or simply had some spare time.

To space my practice, I set reminders on my phone for 1,3,7 and 10 days after learning material. This way, I was making it harder to recall something, but not a time so far apart that I’d have to learn everything again. The goal was to struggle to recall, but not struggle so much that I’d have to re-read everything.

Getting through a trail wasn’t done in ‘one sitting’, but over a series of days. So as I got further along the set of flashcards, and my challenge to recall them, would grow.

Below is a diagram of how I scheduled out spaced repetition and learning tests

Generate ‘real experience’

Using my reminder system, I was able to challenge myself on the high level concepts — after all you can only fit so much onto a flashcard. But of course technical jobs need much more attention to implementation details, so I needed a way to retain information from that too.

Enter the ‘learning test’ — from the book Clean Code4. For those unfamiliar with this term, it’s an excellent approach for developers from ‘Uncle’ Bob Martin. Simply put, rather than reading the documentation for software, use an available testing framework to **prove** what the documentation says.

This was exactly what I did — so for example when I read through a tutorial that said that a Java DateFormat worked in a certain way, I created a TestNG test that would prove (or otherwise) what I was reading.

Tutorial Material

Example: I wanted to see how the Locale and Format arguments worked when provided to DateFormat.getInstance()

Flashcard

Q: How would I retrieve a String of “Jan-01–1970” regardless of where my code was running?
Hint: This is the format used by the UK.

A: I would use the DateFormat .getInstance() Method and pass two arguments
The style would be DateFormat.MEDIUM (int)
The Locale would be Locale.UK (Enum)

Learning Test Examples

@Test
public void whenLocaleUkAndDateFormatIsMediumThenThreeLetterMonthAndDay() throws Exception{
    DateFormat dateFormat = DateFormat.getDateInstance(DateFormat.MEDIUM,Locale.UK);
    assertThat(dateFormat.format(EPOCH_DATE),is("01-Jan-1970"));
}

Example: I wanted to prove to myself that the DateFormat class would use a MEDIUM date format and the Locale of my machine when I provided zero args to the DateFormat.getInstance() factory method

Flashcard

Q: What would happen if you didn’t pass any arguments, to getInstance()?

A: You would get the default DateFormat style of MEDIUM and the Locale of wherever you were running the code.

@Test
public void whenLocaleDefaultAndDateFormatDefaultThenThreeLetterMonthAndDay() throws Exception{
    DateFormat dateFormat = DateFormat.getDateInstance();
    assertThat(dateFormat.format(EPOCH_DATE),is("01-Jan-1970"));
}

This was vital. By running through a series of tests, and seeming them fail, I was able to see that my existing understanding of API’s was wrong in some cases. I could see a test fail, find out from the official docs, or someone’s article and have a test I could run, at will to see the impact.

In some cases, these ‘lower level’ findings were important enough for me to create additional flashcards for, to remind me of something I wasn’t expecting.

Maximise your RoI

I setup an account on bitbucket, so that I could check in any tests that I wrote. Combined with cram being available online and offline (on my tablet and phone), I had reference material that I could refer to anytime, anywhere.

This was huge bang for buck. To give an example, this month I’d written learning tests against the BigDecimal API. At work, when I was under time pressure to see why a unit test failed, I simply pulled down my learning test repo and found a scenario matching what I needed. As I was scanning the code, I got a refresher on the API ‘for free’!

As the ‘Learning How To Learn’ Course has also mentioned5 , each time you recall information in new scenarios, the information and memories are strengthened and enriched, so you’re getting even value out of this approach by having a digital system you can call up.

Final words and findings

I’ve written a more extensive article on how my system was setup, and will link to it to here for anyone interested in specifics in due course. For now I’d like to re-iterate a few key points.

• Its important to decide which subjects warrant this investment because that’s exactly what it is. I’d lean towards this approach for learning the fundamentals of a subject because they are points that are good ‘bang for buck’. You will get the benefits of long term recall for information that will be applied again and again,in a variety of contexts.
• I’m not sure specific implementations of subject matter need to be committed long term — I don’t think the ‘lifespan’ of the is worth slowing you down.
  To give an example, I’d written learning tests about reflection because a Framework like Spring makes extensive use of it, but I wouldn’t write learning tests for developing Spring Web Project at work. In these cases my learning would be from actually developing the project! (And of course, I’d write unit tests for any development I did anyway)
• I was shocked to see how many times an API would surprise me by failing my tests and this nicely demonstrated two points from the book, ‘you learn far more from failing’ and ‘you learn more when you try and generate the answers for yourself’.
• Where you don’t have a concrete use case right now for a fundamental concept, this is where the learning tests really come into its own. You have a set of *proven* reference material with no need to reinterpret documentation each time.

Your feedback

I’d love to hear from you now, in the comments section. Did you like the article? Any pointers for me to improve what I deliver to you? Any success with this approach? Have you hit any stumbling blocks?

It would be great to know what you think.

Resources

Flashcards

As an example of a set of flashcards, I’ve linked to some flashcards I made on reflection, date/times and collections .

Bitbucket project of learning tests

I included a set of resources here that I’ve been using to learn APIs, you’re welcome to use for yourself, but if you re-blog, please link back to this article. Note that some of the tests (collections and concurrency) are still work in progress to get fixed comprehensively — I’ve left them in here deliberately as a reminder that I still haven’t mastered these points.

Citations

1Brown, Peter C, Roedinger III, Henry L, McDaniel, Mark A. “Reflection Is a Form of Practice.” Make it Stick. Cambridge, Massaechusetts, London England: The Belknap Press of Harvard Press, 2014. Pages 26–27. Print.

2https://www.coursera.org/learn/learning-how-to-learn/lecture/BuFzf/illusions-of-competence

3Brown, Peter C, Roedinger III, Henry L, McDaniel, Mark A. “Testing: Dipstick vs Learning Tool.” Make it Stick. Cambridge, Massaechusetts, London England: The Belknap Press of Harvard Press, 2014. Pages 18–19. Print.

4Martin, Robert C, “Exploring Boundaries” Clean Code: a handbook of agile software craftsmanship. Stoughton, Massachusetts. Pearson Education. Pages 116–118. Print.

5Brown, Peter C, Roedinger III, Henry L, McDaniel, Mark A. “The Takeaway” Make it Stick. Cambridge, Massaechusetts, London England: The Belknap Press of Harvard Press, 2014. Pages 43–44. Print.