overview
Finding relevant papers is hard
Researchers want to focus on their work; however, keeping up with the field and finding related papers to what they're currently working on is difficult and time-consuming. My team sought to streamline the paper-finding process in Google Scholar, a widely used search engine for academic literature, by providing tools to help researchers discover and identify relevant papers more efficiently.
user research & insights
"There's too much stuff out there"
We interviewed people of varying levels of research experience - professors, PhD students, Master's students, and undergraduate students - to better all understand and for what reasons researchers find papers, and what challenges they face. We found there are two distinct means for finding papers:
- 1
Searching for papers
Researchers most often use search engines like Google Scholar or SemanticScholar to seek out papers relating to some topic. However, due to the sheer quantity of papers out there, they must adapt strategies to narrow down the search space.
Users reported using various strategies to narrow results to more likely to be relevant papers, such as searching directly within conference sites, which provide a curated set of high-quality, peer-reviewed papers; searching for prominent authors and their coauthors in a given field; or searching for papers related to a notable paper relating to the topic they're interested in.
- 2
Discovering new papers
Researchers also find papers through recommendations from others in academia. All the students interviewed reported receiving paper recommendations from PIs or mentors, and many also reported finding new papers from their lab's workspace (typically Slack). Several people, including two professors, also mentioned using social media platforms like Twitter to find new papers by following other academics in their field.
In total, recommendations from others are useful in that they provide personalized results for little effort - the papers recommended are often pertinent to the researcher's current work, and discovering them requires no searching on the researcher's own part.
redesign
We chose to redesign Google Scholar, a prominent search engine for academic literature, to better aid researchers in finding papers. Google Scholar was the only search engine used by all users interviewed, an in many cases was the sole tool used.
Google Scholar is effective for obtaining a wide range of results; however, it's difficult to narrow down to the results that are actually relevant to what the researcher has in mind. Furthermore, while it captures a detailed activity history of what the researcher has searched for, Google Scholar currently does not provide many options for personalization. Thus, we decided to add two major sets of features to help researchers along the two dimensions of paper searching.
First, we sought to provide various improvements to Google Scholar to better narrow search results, including improved filters, more transparent results, and increased interconnectivity.
Second, to help researchers discover new papers, we added paper discovery feeds into Google Scholar. In particular, we added feeds for personalized content to help researchers discover papers relevant to their work, and trending papers to help researchers keep up with recent developments and discourse in their field.
features
We began by making separate user flows for each of our two major features; afterward, to acknowledge the interconnected nature of our redesign, we combined them to make a single comprehensive user flow demonstrating how they interact.
homepage
All our users in user testing started their Google Scholar paper searching process from the homepage. As such, with the Google Scholar homepage we sought to make use of its large amount of available whitespace to make personalized content easily accessible and discoverable. As such, informed by the Google News homepage, we added a preview of several trending and For You papers for the user, moving the search bar to the top where it appears in all other pages in Google Scholar.
search results
Noting how users often searched for results based on author or venue, we chose to add filters for them in the search results. To provide another heuristic for researchers to decide a paper's relevancy and to provide more transparency to Google Scholar's recommender system, we also chose to add a "match score" displaying the confidence of relatedness of a result with the search query.
interconnectivity
To make it easier for researchers to find or narrow papers relating to some dimension, we broadly sought to make Google Scholar more interconnected by keeping most links internal to Google Scholar-specific representations of content. For instance, each author has their own Google Scholar page associated with them, that in turn provides relevant information about them like their primary fields of study, their papers, and their coauthors; and likewise, each
In total, paper has its own page associated with it, providing more detail about the paper such as its DOI, the full abstract, and links to supplemental materials like its associated code (if any), allowing users to be able to more effectively judge whether a paper might be relevant without having to leave Google Scholar.
All the pages are easily navigable from one another to make it easy to jump to the item of interest: for instance clicking on any author name, whether in the publication details of the paper page or in a search result entry, brings the user to the author's page.
paper discovery
We added a For You page and Trending page to help researchers discover relevant papers. Papers on the For You page are based off of the user's Google Scholar activity, providing personalized results likely to be related to the user's current work; meanwhile, papers on the Trending page are based off general trends of interest and popularity of some paper within the user's field, helping users keep up with hot topics in the field.
iteration
We tested our initial hi-fi prototypes on 4 participants - 2 masters students and 2 undergraduate students - to identify any overlooked pain points or issues and to get feedback on several possible versions of some screens. We iterated on what we found and their feedback to identify the best features from our screens and coalesce them into our final prototype.
homepage
Our initial homepage added a filter dropdown in the homepage such that users could apply filters without having to first go to the search results with a query. However, we found that no user actually used it, even if they did want to filter: they'd typically enter their query (for instance, "2013"), directly into the search bar instead. Notably, one user explicitly declared, "And I completely ignore the filter" while doing so.
With regards to paper discovery, our initial homepage's labels for personalized and trending content were somewhat ambiguous to users, particularly whether they directly correlated with the Trending and For You papers; as such, we renamed the labels to make this connection explicit. Users also wanted to be able to save papers quickly, so that if they needed to search for something else but one of the recommended papers caught their eye, they could get back to it later without having to diverge from or interrupt their current task.
We also found that users wanted to see more papers, so we condensed the page to be able to fit in more recommendations; with making Trending and For You papers on the homepage explicit as well, we looked to the Google News homepage for inspiration, which, in addition to also being under the Google brand and having a search bar, contains separate groups for more personalized and trending news.
For You and Trending pages
Our initial prototype had only a For You page, containing both personalized and trending papers. Users all expressed a desire to have them separate, noting how those two types of papers serve different needs, and that they'd rather have the two separated so they wouldn't have to worry about distinguishing between the two when browsing through them.
Users were also confused by the presence of a match score in the For You page, reasoning that any papers in there should be fairly relevant already. We also removed filtering by followed author the For You and Trending pages based on all users indicating not finding such a filter having much of a use case.
reflection
This project helped strengthen my skills across the design process, and in particular doing so in a collaborative environment. It was also quite interesting to see how we stumbled across a problem that hasn't really been addressed for paper searching - personalized content - when this niche has been at the forefront other, far more generalized contexts (namely, search in general) with products like TikTok built upon personalization. Where else can we draw lessons to help out the research community - whether for paper searching, or other aspects of the research process - or to what other niches can we transfer preexisting ideas and technology to that could bring widespread benefit?
Thanks to Crystal Zhan, Edward De Leon, and Vivian Yan for being awesome teammates and contributing to a culture of both fun and learning; to Megan Tan for mentoring and providing feedback; and to the users who took the time out of their day to interview and user test with us.