Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Hi

Have you looked into these two?

- Trustworthy Online Controlled Experiments by Kohavi, Tang, and Xu

- Statistical Methods in Online A/B Testing by Georgi Georgiev

Recommended by stats stackexchange (https://stats.stackexchange.com/questions/546617/how-can-i-l...)

There's a bunch of other books/courses/videos on o'reilly.

Another potential way to approach this learning goal is to look at Evan's tools (https://www.evanmiller.org/ab-testing/) and go into each one and then look at the JS code for running the tools online.

See if you can go through and comment/write out your thoughts on why it's written that way. of course, you'll have to know some JS for that, but it might be helpful to go through a file like (https://www.evanmiller.org/ab-testing/sample-size.js) and figure out what math is being done.



PS - if you are looking for more of the academic side (cutting edge, much harder statistics), you can start to look at recent work people are doing with A/B tests like this paper -> https://arxiv.org/abs/2002.05670


Even more!

Have you seen this video - https://www.nber.org/lecture/2024-methods-lecture-susan-athe...

Might be interesting to you.


I’ll second Trustworthy Online Controlled Experiments. Fantastic read and Ron Kohavi is worth a follow on LinkedIn as he’s quite active there and usually sharing some interesting insights (or politely pointing out poor practices).


speaking of Georgi Georgiev, I can’t recommend enough his AB testing tools at https://www.analytics-toolkit.com

being able to tell when an experiment has entered the Zone of Futility has been super valuable.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: