Imagine that you are developing a social service "yappy", which is similar to tiktok or…
- User
Imagine that you are developing a social service "yappy", which is similar to tiktok or reels. That is, a short video service.
You are a Data Scientist who helps to improve the recommendation system. Now you have a recommendation system that works like this. The recommendation feed consists of sub-feeds that form a single user feed. The user is sent 15 videos to the device, which are taken from the sub-feeds. For example, 5 videos are taken from the 1st sub-feed, 5 videos are taken from the second sub-feed and 5 videos are taken from the third sub-feed. This is a simplified example. In fact, the sub-feeds have their own names and numbers in the feed.
Before listing the existing sub-feeds, I'll tell you about the SVR ranker.
SVR (Shorts Video Recommender) is a service for making recommendations based on ML (machine learning). SVR acts as a level 2 model that accepts video (candidates) from level 1 algorithms as input.
The task of SVR is to provide relevant personalized videos in response to a feed request.
- The service takes video from an existing algorithm as input
- Runs the video through the ranking model (ML)
- Returns the video in the form of new algorithms
What kind of films are there now:
-
svr_start sub-feed.
The "start" algorithm with ML filtering by SVR methods.
Briefly about the "start"
- The name on the back is "start_new_3"
- All new videos uploaded to Yappy are included in the algorithm.
- The video in the algorithm is a maximum of 72 hours from the moment of publication
- The video gets a maximum of V1 organic impressions. If V1 is scored in less than 72 hours, it is excluded from the algorithm and falls into the “dying pit” — the repository of videos that have passed the "start".
- V1 is configurable. On November 29, 2023 = 30 organic impressions.
- Videos that have not scored V1 organic impressions in 72 hours are also excluded from the algorithm and fall into a dying pit. But then they don't get into the svr_stairs sub-feed.
- The videos are then ranked using SVR for user personalization
-
svr_stairs sub-feed.
This is a sub-feed that works according to the following mechanism.
-
The name on the back is "stairs_new_3"
-
The algorithm includes videos from the "dying pit" that scored V1 organic impressions.
-
Videos go through a "stairs" of stages. At each stage, the video must gain a guaranteed number of organic Vn impressions.
Vn — configurable for each stage. From 30 to 12,500 impressions.
-
After 48 hours from the publication of the video, it will be excluded from the “stairs” after a set of video screenings, they compete with each other for the transition to the next stage.
-
Videos are compared using the ER formula (engagement rate. The coefficients in the formula are configurable).
-
Only a certain percentage of videos uploaded over the past day can go through each stage.
-
Then the videos are ranked using SVR for user personalization
-
After passing the stairs, the videos are excluded from the algorithm.
- "Impressions" – the event when a person saw the video, the beginning of viewing. On the backend, this event is called a "Preview". Views can be counted not only from the "start" feed, but also when viewed from a profile/by direct link/etc.
- L2, L3, L4, L5, L6 – the stages of the video life cycle within the framework of the "stairs" sub-feed.
- V2, V3, V4, V5, V6 – the number of views at each stage that the video receives for selection to the next stage.
- V2 = 40, V3 = 150, V4 = 1500, V5=5000, V6 = 12500
- P2, P3, P4, P5, P6 – the percentage of videos uploaded in the last 24 hours that gets to the next stage.
- P2 = 40%, P3 = 10%, P4 = 2%, P5 = 0.35%, P6 = 0.053%
- ER – engagement rate characterizing video quality
- T is the time period with which the videos fall from the stages into the dying pit. The T parameter is set during development.
-
-
svr_actual sub-feed.
- The name on the back is "actual_recommended"
- The algorithm uses videos published in the last 30 days.
- The video ranking in the algorithm is performed according to the formula ER (engagement rate. the coefficients in the formula are configurable).
- Then the videos are ranked using SVR for user personalization
The position of the video in the Feed for users should be calculated using the final formula, in which the following parameters are applied:
Clean parameters:
- Number of likes: L
- Number of impressions: V
- Number of comments: C
- Number of reviews: R
- Number of screenings: W
Calculated parameters:
- Like rate: LR=L(org)/V(org)
- Comment rate: CR=C/V(org)
- Rate limit: RR=R/V(org)
- Depth of inspections: VTR = W/V(org)
The numerical multipliers L, LR, CR, RR, VTR - in total cannot exceed 1 and must be configured (in Django).
Calculation formula:
- Z = 0.5Norm(L) + 0.2Norm(LR) + 0.1Norm(CR) + 0.1Norm(RR) + 0.1*Norm(VTR) where Norm is linear normalization by [0, 1]
The videos in the feed should be sorted from the largest to the smallest by the Z parameter
-
Subscription subscription sub-feed.
Videos from new to old by date of publication, from the authors to whom the user is subscribed.
This feature is available only to registered users, because only they can subscribe to the authors.
-
prunc sub-feed.
Extract video embeddings from video with X-CLIP with microsoft model.
Recommend similar on user history. E.g. user liked some videos, prunc recommend similar on this videos.
Similar video obtains from Qdrant ANN search database. In this databases stores only those videos who gets more than 20 likes.
There is no SVR ranking.
Question: come up with 10 hypotheses on how to generate candidates with heurisitcs. Now we are doing this with the help of sub-feeds such the svr_stairs, svr_actual, svr_start, subscribers, prunc subs. Suggest 10 heuristics for enriching candidate generation. Give comprehensive description for every heuristic. Explain why they will work. How calculate every heuristics. Example of heuristics:
- top authors. Authors which get the biggest amount of likes and subscribers. (ER). Add high quality content to candidates. Calculate likes for every author. Calculate subscribers of every author. Calculate weighted coefficient of every author (0.3likes + 0.5subscribers)
Assistant
Here are 10 additional heuristics I could suggest for generating video candidates in the Yappy recommendation system:
-
Rising stars - Identify authors whose videos have recently started getting much higher than usual engagement. Add their latest videos to candidates. This captures potentially viral content early. Calculate a 14-day ER trend for all authors and select ones with sharp increases.
-
Author similarity - For each video, find authors with content style/topics closest to it based on co-engagement, shared hashtags etc. Add those similar authors' recent videos to candidates. Helps recommend videos to niche interests.
-
Fresh backlog - For each user, add videos > 1 month old from authors they follow which they haven't seen yet to candidates. Handles returning users with updated content.
-
External events - Add recent videos that align with major external events (sports, politics) to candidates. Maximizes chances of recommended content being relevant. Identify events via news APIs.
-
Video title embeddings - Encode video titles with sentence transformers, find closest titles to a user's engagement history. Helps surface overlooked relevant content.
-
Co-engagement graphs - Build graphs between videos based on user co-engagement. Add videos close to those a user recently engaged with to candidates. Capture themes.
-
Geography trends - Identify recent viral video themes by geography, add related videos to candidates for users in those geographies. Reflect local tastes. Cluster geo-located engagements.
-
Fresh hashtags - Track rapidly emerging hashtags. Add recent videos featuring those fresh hashtags to candidates. Quickly recommend trends. Calculate hashtag momentum.
-
User comment co-occurrence - Find videos frequently commented on by same users commenting on a user's engaged videos. Assumes interest overlap. Text analysis on comment text.
-
ER volatility - Add videos with recent high ER variance to candidates, capturing potentially viral growth. Calculate standard deviation of daily ER over past week per video.
-
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.