Hulu DevX Console

UCI Capstone Project

Summary

Tasked with helping Hulu engineers simplify their fragmented workflows, a group of 4 including myself designed a new dev console for them to leverage as our capstone project through UC Irvine.

Role + Team

I worked with 3 other students from my cohort to complete this project. I focused mainly on design and prototyping, but I was actively involved in research and testing as well.

Tools

•Figma

•Mural

•Validately

•Zoom

Executive Summary

Behind the scenes, the DevX Team at Hulu plays a key role in building tools for developers and ensuring that everything runs smoothly. In order to complete their daily tasks, the DevX Team juggles numerous tools per day, and sifts through thousands upon thousands of documents related to those tools. The team was in need of a simplified, more efficient workflow–and that is where we came in.

In this project, we consolidated documentation, provided an agile way to switch between tools, and created an onboarding process for new hires–all in one unified web portal.

Methodology

Over 6 months, we utilized a double diamond methodology in which we thoroughly researched and understood our users, synthesized and defined our findings, and translated them into visually effective deliverables for Hulu to leverage moving forward with development.

Discover: Research

During our capstone kickoff, my group and myself immediately recognized how important effective research would be in successful completion of this project. None of us were software engineers and we had never worked on a team like DevX. We quickly devised a plan and sent out a survey to get a head start on our data gathering. My teammates Hannah and Yu lead research initiatives, but myself and the other designer were heavily involved with research too.

Initial Survey and Bubble Charts

Following kickoff, we sent an initial survey out to the DevX org at Hulu to see what types of tools they used and if they had any ideas on what they would like to see in this new console. We also asked participants if they would be open to interviews or shadowing sessions in this survey to prepare for the next research steps.

We created bubble charts of the tools each team used within the DevX organization to get a feel for what types of UI patterns they were used to and what their workflows looked like.

Other Key Survey Findings

80% of participants had used a dev console.

Easy searchability was the most requested feature from our users.

Users also wanted consolidated documentation and resources.

New Hulu employees are called "Nulugans."

Nulugans faced a learning curve with Hulu's internal tools.

4 participants agreed to virtual shadowing.

8 participants agreed to user interviews.

Virtual Shadowing and Interviews

We shadowed 4 DevX engineers to gain a thorough understanding of what they did in their day to day workflows. We learned they used quite a few tools to accomplish their work and also bounced back between a lot of different resources to sift through documentation.

We interviewed the participants we shadowed plus a few more. Leveraging questions informed from the shadowing sessions, we dove into some areas we still wanted to clarify like their workflows, what tools they liked using and why, and the most important features and tools they expected in a new console.

Key Shadowing Findings

Users relied heavily on search.

Bookmarks were popular amongst participants for locating tools and documentation.

Some participants just remembered the URLs they frequently referenced instead of bookmarking.

Participants used at least 8 different tools during a 1 hour shadowing session.

Stakeholder Presentation of Research

After collecting our research, we presented our findings to our stakeholders at Hulu to get their feedback. Overwhelmingly, our stakeholders were very positive about our research findings and were encouraging regarding our direction moving forward.

"I almost feel emotional, it's like our needs are finally being heard."

-Technical Product Manager, Hulu

Key Interview Findings

Addressing fragmented and unreliable documentation was a big priority for users.

Hulugans are very accustomed to Apple design patterns.

Users favored efficient and uncluttered UI with rigorous search features.

Shortcuts were very popular when navigating workflows.

Define: Synthesis

With the blessing of our stakeholders, we advanced to the second half of the first diamond to synthesize our findings and start coming up with a design direction.

User Personas

Our first task was to create user personas. Based on our shadowing and interview sessions, we made 3 personas to represent the types of user in our research: Nulugans, Seasoned Software Developers, and Seasoned Team Leads.

User Journey Maps

Journey Map - Seasoned Software Develope

How Might We Questions

After getting to know our users a bit more through personas and user journey maps, we started ideating ways our console could help them work more efficiently through how might we questions. We did this as a group in Mural.

Narrowed Down

Synthesizing a Design Direction

Through research, we discovered many pain points with our users, but we wanted to focus on effectively solving a few problems to not blow up the scope of this project.

After our how might we brainstorming, we decided to focus on three core experiences in our console:

1. Nulugan Onboarding

2. Tool Switching

3. Documentation Consolidation

Card Sort + Impact/Effort Matrix

As a final step before jumping into sketching and wireframing, we conducted a card sort to glean the most important features to our users and organized the findings into an impact/effort matrix.

Develop: Sitemap, Wireframes, and Testing

Jumping into the second diamond, myself and another designer took charge of starting to sketch and wireframe our ideas so we could start evaluating them with our users.

Information Architecture – Site Map

We put together an initial site map to organize the information architecture of the console and get an understanding of how a user would navigate it.

Concept Sketches

We then created sketches using procreate to visually explore some ideas we had for core features.

LoFi Wireframes

We put together a LoFi wireframe prototype to bring our initial ideas to life. We focused on onboarding, the service dropdown, documentation, and the home page with this prototype to test out and get feedback on some of the core problems we were solving for.

Click below to see the prototype we built and tested using Figma and Validately.

View LoFi Prototype

User Testing – LoFi

Using Validately, we conducted 4 user testing sessions that lasted 30 minutes each with members from 3 teams within the DevX organization. We also sent out a prototype link and questionnaire to the DevX organization, which we collected 9 responses from.

We focused on evaluating onboarding, the home page, and the documentation section. Below are some quotes and key findings the first round of testing provided:

"Looks great, much more consistent with Hulu design language."

"Simple, sleek, clean"

Usability Feedback

Onboarding was light, simple, and not overwhelming. They wanted more information about tools though.

Featured content on the home page was very popular and users loved it.

They wanted bookmarks or an easy way to pin documentation they liked.

They found the initial services dropdown a bit confusing and questioned the IA.

Iterating to HiFi

We met as a team following the first round of testing to discuss the feedback and what the path forward looked like. We made some key changes to some pages like the service dropdown to create a more user-centered approach and also sharpened the design up to high fidelity.

View HiFi Prototype

User Testing – HiFI

We conducted a final round of user testing with our HiFi prototype. This time, 5 developers from four different teams participated in hour long sessions to evaluate the whole experience.

"Maybe it'll become my new homepage!"

"It seems like the dev console would make my life easier."

Usability Feedback

Users found the onboarding "friendly" and "welcoming."

Users liked the inclusion of personal pronouns in onboarding

Users wanted tags or labels on documentation to better determine its relevancy

This version of the service dropdown was much better received.

Deliver: Final Prototype and Key Features

To finish off our project, we incorporated the feedback from our last user testing session and also met with our stakeholders to get final approval before presenting and delivering our solution.

View Final Prototype

Key Features

Service Dropdown

The service dropdown offers an easy way to access all the documentation, guides, and important links in the dev console. It's accessible from the top nav and was seen as a critical feature for this product.

Settling on an intuitive but robust IA was a challenge in this project, but with aggressive research and testing, this layout proved the best option.

Users especially loved the estimated time callout on this page for guides.

Search + Tags

One thing we observed and heard in research was how important search was to our users. We paid a lot of attention to our global search screen to make sure it presented the information our users were looking for and added filtering as well.

Through testing, we uncovered a key need for tagging content across the console. We leveraged this not only on guides and documentation but also on individual user profiles so user could easily flag down subject matter experts.

Tool Mapping

When we started this project, we saw tool mapping as something that would be cool to have but definitely a cherry on top type of feature.

In testing, one user in a leadership role stated that tool mapping could be the most valuable feature in this console, so then we made sure to focus on it more.

Tool mapping is very important for users to understand how the many tools they use interact and understand their value. Developers at Hulu use a ton of tools, so this understanding was critical to their success.