Oct - Dec 2022
Product Design
AI and Machine Learning
01.
Starting from Oct 2022, I joined Scale AI as a product design intern. Scale is a SF unicorn startup founded in 2016, their product accelerates the development of AI applications by helping machine learning teams generate high-quality data, focusing on building differentiated models vs. labeling data. I worked closely with my manager and product team, we had daily check-ins and also participated in weekly design critique with the design team. I presented my intern project on week 12, in front of my peer interns and teammates.
My team is called Content Understanding, which is a product improves platform experiences by enriching content metadata, discovering trend insights, and flagging sensitive content. I led the end-to-end design for both Content Intelligence and T&S(Trust and Safety), which has been shipped and successfully demo to our clients and on Transform X, a three-day conference viewed by millions of attendees.
02.
Content forms are ever-evolving and growing as media consumer technologies advance. Enrich their content’s metadata to keep up with the growth of the content, provide the highest quality recommendations, and keep users entertained and engaged.
03.
The Taxonomy Management Platform is a product meant to make it easy for customers to view their taxonomies, request changes, and then see those changes through. It is an interactive platform for Scale’s enterprise customers.
Depending on the requirement, Scale can be flexible on what we can provide across several dimensions. In this specific project, the customer send us a set of short-form videos from US producers and generates three kinds of labels:
• Content topic based on existing taxonomy
• Does the video contain the content creator
• Scale’s topic taxonomy and quality signals
After we align on specific POC quality SLAs + taxonomy with key stakeholders, Scale proposed pilot business terms and final timeline to the customer. In this case, the taxonomy management design tool is essential for our customer.
I started with a deep understanding of Scale’s Infrastructure and how could we fit into the space.
For User
• Increase engagement:
- views
- like/comment
- shares
• Generate more content on their platform
• Improve quality of recommendation
• Increase accuracy of ad targeting
• Increase conversion
For Scale
• Provide flexibility to customers
• Lower cost
• Easy integration
• Tight feedback loop
For P0, we have the following functional requirements that needs design supports:
Taxonomy editor tool
1. Request Changes: User should be able to view version history and edit nodes.
2. Request Relabeling: user has the ability to denote which task they want to be relabeled.
3. Request Quick Review: User can review 5 example tasks for each change.
User Story:
My hashtag data analysis is telling me that “dalgona coffee” is trending right now. I recently requested for the taxonomy to add that as a label, and to also remove “Met Gala” from the tree because the Met Gala happened 4 months ago.
Painpoints:
1. I want to see what the current taxonomy look like and how does it compare to the old version?
2. What are some of the examples of the labels? I want to make sure the label is being captured properly.
3. Where can I request more changes? My hashtag data analysis is now telling me new trending labels. I want to add this label.
04.
How might we help user navigate through the tool and find what they want?
After understanding user's pain points, we started exploring ideas based on generated user stories. How might we surface the information to be discoverable?
Taxonomy mangement tool infrastructure
We did usability testing with the internal team and have them complete the task without much guidance and decide on the one causing the least friction. After the early design iterations, we were able to identify some key insights into how users would use the product to find what they need.
Request changes
request relabeling
Initially the idea was to have Scale labeling team automatically label five videos for quick review and let customer choose the one that they want to do quick review on. I interviewed with teammates and realized this process might cause unnecessary work load on Scale side, so I proposed with the idea to let user select specific path that they want better alignment, not only making this step more intentional, also cut down significantly on our internal labeling team.
request quick reviews
quick reviews
05.
We shipped the first version of taxonomy management home and demo it to our customer to gather more feedback. Here is a walkthrough of the final design.
Request changes
Request Relabeling
request quick review
.gif)
quick review
As we shipped the first version of the product, we should also take certain metrics into consideration. Some of the potential criteria include:
For User
• Engagement lift after taxonomy change
• Increase in number of new contents after taxonomy change
For Scale
• Time to completion for taxonomy change before and after
• Cost reduced
• Ease of integration
• Steps reduced
• Survey NPR
For Taskers
• Reduce time per task
• Increase on tasker accuracy
-i.e. does tasker accuracy decrease significantly when taxonomies are evolving
The first couple weeks at Scale were intense, not only to finish onboarding training and started my first project on week 1, I was given a diverse range of problems and tasks that requires new domain knowledge. Designing for non-technical users requires empathy and a deeper understanding of the problem we are solving, focusing on making user-centric design decisions. At the end of the internship, I made huge progress in handling ambiguity in design, developed innovative designs that meet the need of the product and received positive feedback from stakeholders about my work.
Product Design team
