At Academia.edu, our subscription revenue had historically come from our Author tools - ways for scholars to track the impact of their work. Our team was tasked with uncovering new lines of business, and we turned to the millions of users who weren't authors - Readers.

By running hundreds of experiments - often dozens concurrently - our team was able to build two new products - Bulk Download and Paper Summaries - that we launched and iterated on (and a few we launched and scrapped). Above is the single work page, including three premium reader tools: PDF Package/Bulk Download, Translation, & Paper Summaries.

Our work began with an 2 week sprint focused on launching in-paper "keywords," attempting to create a psuedo-wikipedia-pages, focused on unique keywords within a paper, founded on our understanding of users wanting a deep dive on a topic. After launching and tracking usage, we found a peculiar behavior emerging - a user would click into a keyword, and without opening any papers, would download everything associated with it. That led us to believe that the real opportunity was to design a download workflow, rather than a pure browsing flow.

One of our test that stands out most to me is the impact of the paper count - we tested ranges from 15 to 100, and for some reason, conversion lifted by nearly 47% when we served up 37 papers. I'm still curious about that one.

That led us to the concept of collections, or curated bundles, that a user could browse and sample at will. Unlike the machine generated PDF Packages above, these were human curated, and often including supplementary texts and introductions. We couldn't find a way to scale these effectively, so we shuttered the concept, but our learnings here helped inform future Academia.edu products, such as Video Courses and Academia.edu Journals.

One additional experiment focused on using a combined ML + ElasticSearch query to identify unique phrases in the nearby paper cluster in order to quickly answer "what makes this paper special?" In 2 weeks, we build a proof of concept that summarized papers section by section (in a pre Open-AI world, too!). Then, we took our prototype to the nearby Berkeley campus and guerilla recruited students and faculty to test our work. We walked away with dozens of suggestions, shown below, which provided us a combined algorithm of human heuristics + machine identification.

All in all, we went from concept —> market for 4 products in 2 months: Keywords, PDF Package, Collections, & Paper Summaries. We shuttered two - keywords & collections - and doubled down on the others, shifting our energies to testing and optimizing our funnels. A few examples of our test top of funnel are below.

Through these experiments & others, we increased our annual premium subscription run rate nearly 2x. These two products became top-five converting products within 3 months of launch, helping us achieve our goal of 1,000,000 premium subscriptions.