Signal Boosts for June 2024
reading time:   ·   by nicky case

Ay it's yo' girl Nicky, sharing another collection of cool/useful stuff I found last month!

In no particular order:

🌱 Khan Academy's tutor-bot

fan of the app, not a fan of the Cronenberg-esque fleshy-speech-bubble mascot...

First off, I'm not being sponsored by Khan Academy or OpenAI to mention this. I mean, I'll take your money, but no-one's offering.

Anyway: 1-on-1 tutoring's been long known to have one of the greatest effect sizes on learning[1], but alas, 1-on-1 tutoring ain't cheap. So, that's always been the promise of AI tutors: personalized tutoring, but affordable to all.

So when ChatGPT first came, I was excited then immediately disappointed, coz wow ChatGPT sucked at tutoring math / physics / technical topics. For example, I asked it to help me re-derive the rocket equation (which tells you how a rocket's velocity changes) and after 20 minutes of stringing me along with plausible BS, it told me, "assume the rocket's velocity is constant". That's when I closed the app in disgust.

Well 2 years later, I can finally honestly say I've experienced an actually good AI tutor:

Khanmigo, by Khan Academy! (powered by GPT-4)

(Ok, I haven't tried the full Khanmigo yet, I'm currently using Khanmigo Lite on a free plan of ChatGPT. But after a month of trying it, I'm excited to buy the full version!)

Khanmigo, so far, has helped me understand the rocket equation, some integral calculus, and misc stuff from math & physics I wanted to buff up on.

A few features of Khanmigo:

Of course, Khanmigo's missing that social human element[2], and it's not as good as a master teacher like, say, 3Blue1Brown... but dare I claim, it's on par with an average college tutor, at only $4/month![3] Also, it's free for US teachers, and Khan Academy's a non-profit.

And yes, since I'm someone who's entire income comes from making educational stuff, I should be slightly scared of this. Oh well. Not a bad way to end up broke, I guess.

πŸͺ¨ Obsidian, a note-taking app

I've been on the hunt for a good note-taking app for years. A decade ago I used Notational Velocity, but my needs outgrew its minimalism. Then I used Bear but it doesn't have folders & crashed a lot on tablet. Then I used Notion but dear god it's so slow & laggy, also there's no offline mode on Android? Then I used Google Keep, but again, no folders and sometimes laggy. Then I tried a lesser-known app called MWeb, which was pretty good, but there's no Android version.

This month, I (re-)found Obsidian! I actually did try Obsidian last year, but it didn't stick because I went overboard with their cross-linking note system & made a mess of it. But now, I use Obsidian for almost everything text-related:

Some features of Obsidian that make it especially good for me:

I currently use Obsidian on Mac OS X, iPad, and Android. App itself is free, but I pay $4/month for Obsidian Sync, to synchronize all my devices. Again, I'm not being sponsored to say any of this, I just really like Obsidian.

Check out Obsidian here! Available on all major platforms.

(CORRECTION on June 29, 2024: The original version of this post claimed that Obsidian was open-source. It's not. I don't know how I mixed it up in my head, sorry. Sincere thanks to J.C. & Q for emailing me about the error!)

🍸 Data Colada, data detectives

A previous version of this post was supposed to go live in January 2019. But the day before it was scheduled, the Data Colada team (Uri, Leif, and Joe) received an email that we took to be a potential death threat. After discussions with the local police, the FBI, and our families, we decided to postpone its publication indefinitely.

~ from Data Colada #117 [emphasis added]

That wasn't even the first time these data detectives got seriously threatened. From Data Colada #116: [emphasis added]

We identified evidence of fabricated data in four studies co-authored by Francesca Gino. We asked her employer, Harvard University, to look into it. They spent a year and a half doing that, and determined that we were right. So they contacted the relevant journals with requests for retractions and put Gino on unpaid administrative leave. Then we wrote some blog posts about it. Then Francesca Gino filed a $25M defamation lawsuit against us and Harvard. Legal wackiness ensued.

That lawsuit is still ongoing.

Anyway, what I love most about Data Colada's posts is they genuinely feel like murder mysteries. Showing exhibit A, exhibit B, etc, all independently pointing to stinky fishiness.

For example, in one of their biggest fraud busts [Data Colada #98], they investigated a famous study on honesty. (Yes: Fraud, in a study on honesty. Lady Irony must eat well, coz irony be thicc.)

Supposedly, the study asked 13,488 folks to report how many miles they drove. Now, if the data was real, you'd expect a bell curve like this:

What you would NOT expect, is the data looking like this:

That is: looking like someone typed RANDBETWEEN(0,50000) in Excel to get uniformly-random fake data. And this was just one of the many signs of fraud our detectives uncovered.

Anyway, to read their "data murder mysteries", check out the above links! Data Colada also writes insightful articles on statistics in general. One of my faves is: why the standard way to detect publication bias sucks, and a better way to do it. (the authors invented a new statistical tool!)

πŸ”ͺ Holly Jackson's mystery novels

Iiiiiiiii just binged on all (but one) of Holly Jackson's books last month.

My mini-reviews: (90% spoiler-free)

(If you hate reading, there's a BBC adaptation of the first novel coming out July, then streaming on Netflix in August. I probably won't watch it.)


True fact!

Anyway, that's all for this month, folks. Go home.

  1. An oft-cited old study found that 1-on-1 tutoring + "mastery learning" = a two-sigma improvement: that is, the average tutored student gets better than 98%(!!) of non-tutored students. (or, going from a C student to an A student, according to this grade curve calculator)

    Alas, like much early psych research, this doesn't replicate. (See these reviews )

    According to recent meta-analyses, 1-on-1 tutoring's effect size is ~0.5, and mastery learning's is ~0.5, so added together they probably "only" have a 1-sigma effect: taking an average student to above 68% of other students. (or, C student to B student)

    Which definitely isn't as impressive... but still pretty good! β†©οΈŽ

  2. Actually, the "social human element" may not matter much: "No significant difference between learning from [tutor software] and learning from individualized human tutoring or small-group instruction."

    But I still believe (without hard evidence) that an excellent human (like 3Blue1Brown) can still out-teach the best current software (for now). β†©οΈŽ

  3. For context, the average hourly tutor rate is ~$20. So, instead of 1 hour of an average human tutor, you could have 5 months of Khanmigo. β†©οΈŽ

  4. From the Innocence Project: β€œIn approximately 25% of the wrongful convictions overturned with DNA evidence, defendants made false confessions, admissions or statements to law enforcement officials.” False confessions happen due to a subject's intellectual disabilities, coercive/deceptive interrogations, being bribed with promise of immunity, sleep-deprived confusion, to protect someone else, etc. Other replicated research shows it's really easy to elicit false confessions, even with negative consequences. β†©οΈŽ