Skip to content
TheChatArchiveThe Chat Archive
Step 3.5 Flash· unknownHugging Face Frontier Reasoning Traces

Engineer a language-agnostic code ownership inference engine using graph embeddings:…

  1. User

    Engineer a language-agnostic code ownership inference engine using graph embeddings: training data, validation, bias mitigation, and integration plan.

  2. Assistant

    We are designing a language-agnostic code ownership inference engine. The idea is to use graph embeddings (likely from a graph representation of code and developer activities) to infer which developer is most likely to own a given piece of code (file, function, etc.). We need to consider:

    1. Training Data: What data do we need? How to collect and label it?
    2. Validation: How to evaluate the model? Metrics and splits.
    3. Bias Mitigation: How to avoid biases (e.g., towards senior developers, certain teams, etc.)?
    4. Integration Plan: How to integrate this into a development workflow?

    Let's break down each part.


    1. Training Data

    We need a dataset that maps code artifacts (files, functions, classes, etc.) to developers (owners). We can build a graph where nodes represent code artifacts and developers, and edges represent relationships (e.g., developer committed to artifact, artifact modified by developer, etc.). Alternatively, we can represent code artifacts as nodes and use developer activity as features, but graph embeddings typically work on graphs.

    Graph Construction:

    • Nodes:

      • Code Artifact Nodes: Represented by their content (or features extracted from code) or by their unique identifiers (e.g., file paths, function names). For graph embeddings, we need to have a way to generate initial node features. This could be:
        • For code artifacts: Use static code features (e.g., language-specific ASTs, tokens, or even raw code embeddings from models like CodeBERT, but note language-agnostic requirement). Alternatively, we can use simpler features: metrics like lines of code, number of functions, imports, etc. (but might be language-specific). A more language-agnostic approach: use textual features (e.g., bag-of-words from code) or use a universal code embedding model that works across languages (like a multilanguage code model).
      • Developer Nodes: Represented by their identity (e.g., email, ID). Initial features for developers? Could be aggregated activity metrics (total commits, tenure, etc.) or one-hot (but then we need to combine with code features). Alternatively, we can start with no features for developers and rely on graph structure.
    • Edges:

      • Commit Edges: Between a developer and a code artifact if the developer has committed changes to that artifact. Could be weighted by recency, number of commits, etc.
      • Temporal Edges: We might also consider edges between code artifacts (e.g., if two files are often changed together in the same commit, they are connected). This might help with propagation.

    We might build a bipartite graph (developers and code artifacts) with edges for commits. Then we can use graph embedding techniques (like Node2Vec, DeepWalk, or GraphSAGE) to learn embeddings for both developers and code artifacts. Then, for a given code artifact, we can find the nearest developer node in the embedding space.

    Labeling:

    We need to assign an "owner" label to each code artifact. This can be derived from:

    • CODEOWNERS file (if exists) in the repository.
    • Heuristic: the developer who made the most commits to the file (or the most recent commit) as a proxy.
    • Alternatively, we can use a combination: define ownership as the developer who is responsible for reviewing/approving changes (extracted from pull request data). But that might be harder to get.

    We need a large dataset from multiple repositories to ensure generalization. We can use open-source projects from GitHub (with permissive licenses). For each repository, extract commit history and (if available) CODEOWNERS. We can also use the Git history to see who typically reviews/approves changes (via pull request data from GitHub API).

    Data Collection Steps:

    1. Select repositories (maybe from GitHub with a query for those having a CODEOWNERS file, or those with a large number of commits and multiple contributors).
    2. For each repository:
      • Clone the repo and get the full commit history (using git log).
      • Parse the CODEOWNERS file to get official owners for file patterns.
      • Also, extract for each file (or function, if we want finer granularity) the set of developers who have committed to it and their commit counts, last commit date, etc.
      • Possibly pull request data to see who reviews/approves changes.
    3. Build a graph per repository? Or combine all repositories into one big graph? Since we want language-agnostic, we might combine multiple languages. But we need to handle the code artifact features in a language-agnostic way.

    Handling Code Artifact Features:

    We need to represent code artifacts in a way that is meaningful and language-independent. Options:

    • Simple metrics: For each file, compute:

      • Lines of code (LOC)
      • Number of functions/classes
      • Complexity metrics (cyclomatic, etc.) - but these are language-specific? We can use language-agnostic tools like lizard that support many languages.
      • Import/dependency count (maybe using include statements, but again language-specific). Could use a generic parser for each language? That complicates.
      • Alternatively, treat the code as text and use a pre-trained language model (like CodeBERT, which supports multiple languages) to get an embedding for the entire file. But note: CodeBERT is trained on multiple languages, so it can produce embeddings for code in different languages. However, it might be biased towards languages it was trained on more. But it's a good starting point.
    • Graph features: If we also connect code artifacts based on co-change (files changed together), we can use graph embedding techniques that propagate information along these edges. Then the initial features for code nodes could be just a constant (like 1) or a one-hot for the file type (extension). But then the embedding will capture structural role.

    We can try both: start with simple initial features (like a constant) and rely on the graph structure (commit and co-change links) to shape the embeddings. Since we have a bipartite graph, the embeddings will be influenced by the developers they are connected to.

    Target Variable:

    We want to predict the owner(s) for a code artifact. This is a multi-label classification (a file can have multiple owners). But we might simplify to single owner (e.g., the primary owner) for training. Or we can train to predict a distribution over developers.

    We need to decide: for each code artifact node, we have a set of candidate owner developers (maybe from CODEOWNERS or commit history). We can treat it as a link prediction problem: given a code artifact node, predict which developer node(s) are connected to it (or should be connected). But during training, we have existing commit edges. However, ownership is not just about who committed; it's about responsibility. So we need to define the "ownership" edge as a separate edge type? Or we can use the commit edges as positive examples but also need negative examples (developers who are not owners). That's tricky.

    Alternatively, we can formulate it as a classification problem: for each code artifact, we have a set of possible developers (maybe all developers in the repository? That's too many). We can limit to developers who have ever committed to that artifact? But then we are only ranking among those who have touched it. But ownership might include people who haven't committed but are responsible (like maintainers). So perhaps we use CODEOWNERS as ground truth, and we want to predict that set. Then we need to generate negative examples: for each file, we have a set of positive owners (from CODEOWNERS) and we can sample negative developers (from the set of all developers in the repo, excluding the positives). But that's a lot of negatives. We can treat it as a binary classification per (file, developer) pair? That might be heavy.

    Better: we can learn embeddings for code artifacts and developers, and then for a given code artifact, compute similarity (dot product) to all developer embeddings, and then rank them. Then we can use a loss function like contrastive loss or binary cross-entropy with negative sampling.

    So we can define a training instance as (code_artifact, developer) with label 1 if that developer is an owner (according to CODEOWNERS) and 0 otherwise. But note: CODEOWNERS might list a team or a pattern, not an individual. We might need to map teams to individuals (maybe from GitHub teams). That's another complication.

    Alternatively, we can avoid relying on CODEOWNERS and use a heuristic: the owner is the developer who has made the most commits to the file (or the one who created it). But that might not reflect actual ownership (people move, ownership changes). However, for training data, we can use the current CODEOWNERS as ground truth if available, and for repos without CODEOWNERS, we might not have ground truth. So we need a way to label.

    Maybe we can use multiple signals:

    • CODEOWNERS as primary ground truth (if present).
    • For repos without CODEOWNERS, we can use the developer with the most commits (or the most recent commit) as a proxy. But that's noisy.

    We could also use the "reviewer" data from pull requests: if a developer frequently reviews changes to a file, they might be an owner. But that's harder to get at scale.

    Given the complexity, maybe we start with a simpler approach: use commit history as the only source of ownership. Define ownership as the developer who has made the most commits to the file (or the most recent commit) and treat that as the single owner. Then we have a single-label classification problem: predict the top developer for a file. However, that might not be accurate because ownership is not just about quantity.

    But for an inference engine, we want to predict who to assign as owner for a new file or when ownership is unclear. So we need a model that captures the relationship between code and developers based on past behavior.

    Perhaps we can frame it as a recommendation problem: given a code artifact, recommend a set of developers who are likely to own it. We can use the graph embeddings to compute similarity scores.

    Graph Embedding Model:

    We can use a method like TransE (for bipartite graphs) or Node2Vec to embed both code artifacts and developers into the same vector space. Then for a code artifact, we find the nearest neighbors among developer nodes.

    Alternatively, we can use a graph neural network (GNN) like GraphSAGE to learn embeddings by aggregating neighbor information. Since we have a bipartite graph, we can treat developer and code nodes separately.

    Steps:

    1. Build a graph G = (V, E) where V = {code artifacts} ∪ {developers}.

    2. Edges: (developer, code) if developer has committed to code. We can have multiple edges (over time) or aggregate into a single weighted edge (e.g., number of commits).

    3. We might also add edges between code artifacts if they are changed together (co-change). That would create a separate type of edge. But then we have a heterogeneous graph? Actually, we can treat all edges as the same type (undirected) or have different edge types. Simpler: treat all as undirected and unweighted? Or weight by commit count.

    4. Run Node2Vec on this graph to get embeddings for all nodes.

    5. For a given code artifact, compute cosine similarity between its embedding and all developer embeddings, and rank.

    But Node2Vec treats all nodes equally, and the graph is bipartite, so the embeddings will separate into two clusters? Possibly. But we can also use Metapath2Vec for heterogeneous graphs if we have multiple node types. But Node2Vec can work on homogeneous graphs if we just mix nodes. However, the graph is bipartite, so random walks will alternate between developer and code nodes. That's fine.

    Training Objective:

    We can train the embedding model to predict which developers are likely to be connected to a code artifact (i.e., have a commit edge). But we want ownership, not just any commit. We have a label for ownership (positive) for some edges (maybe we only consider edges that correspond to ownership? But commit edges are not necessarily ownership). This is a problem: we want to infer ownership, but we only have commit data (which is a superset of ownership? Not exactly: owners commit, but not all committers are owners). So we have noisy positive examples if we use all commit edges as positive. And we have no explicit negative examples (developers who are not owners but might have committed). Actually, if we treat ownership as a subset of commit relationships, then we have some edges that are positive (ownership) and some that are negative (commit but not owner). But we don't know which commit edges correspond to ownership. So we need to use CODEOWNERS to label some edges as ownership. That gives us a subset of edges (or pairs) that are positive for ownership. But note: CODEOWNERS might list patterns, so for a given file, we know the owners (developers or teams). We can treat each (file, owner) as a positive example. For negatives, we can sample random developers who are not owners for that file.

    So we can create a dataset of triples: (code_artifact, developer, label) where label=1 if developer is listed as owner in CODEOWNERS for that artifact (or pattern matches). Then we can train a model that takes the embeddings of code and developer and predicts the label. But then we need to first get embeddings. We could do two-step: first get Node2Vec embeddings, then train a classifier on top. Or we could jointly train: use the graph structure and the ownership labels as a supervised signal. That would be better: we can use a GNN with a supervised loss on the ownership labels.

    So we can build a graph where nodes are code artifacts and developers. We have commit edges (unsupervised) and we also have ownership edges (supervised, a subset of commit edges? Not necessarily: an owner might not have committed yet? But typically they have. But CODEOWNERS might list someone who never committed? Unlikely. But we can treat ownership as a separate edge type? Or as a node label? Actually, we can treat it as a link prediction task: given a code artifact, predict which developer nodes are connected via an "ownership" edge. But we have only positive ownership edges (from CODEOWNERS). We don't have negative ownership edges. We can generate negatives by corrupting (code, developer) pairs that are not in CODEOWNERS but might be connected by a commit? That's messy.

    Better: we can treat it as a multi-label classification on code artifact nodes: each code artifact has a set of owner developer nodes. Then we can use a GNN that takes the graph (with commit edges) and node features, and for each code artifact node, outputs a probability distribution over all developer nodes? That would be a huge output space. Alternatively, we can use a decoder that scores (code, developer) pairs. That's like a knowledge graph embedding (e.g., TransE, DistMult) where we have relations: (code)-[owned_by]->(developer). Then we can learn embeddings for code and developers such that for positive pairs, the score is high. We also have commit edges as another relation? That would be a multi-relational graph. We could have two relations: committed_by and owned_by. Then we can use a relational GNN or a translational model.

    But note: we want to predict owned_by from the graph structure and maybe node features. So we can train a model that, given the graph with committed_by edges and node features, learns to predict owned_by edges. That's link prediction on a single relation, but we have other edges present.

    Simpler: we can build a graph with only one edge type: undirected edges between developers and code artifacts if there is a commit. Then we have some positive ownership pairs (subset of these edges? Actually, ownership might be a subset of commit edges, but sometimes an owner might not have committed yet? We can assume that if someone is listed as owner, they have at least one commit? Not necessarily, but we can require that for training data: only consider owners who have at least one commit to that file? That might be reasonable. So we can label some of the commit edges as ownership edges (if the developer is in CODEOWNERS for that file). Then we have a binary classification on edges: given an edge (developer, code), is it an ownership edge? But then we need to consider edges that don't exist? We want to predict ownership for a code artifact, which is essentially predicting which developer nodes are connected via an ownership edge. That's link prediction. We can train a model that scores existing edges (positive) and non-existing edges (negative) but we only have positive edges for ownership. However, we can use the commit graph to generate negatives: for a given code artifact, we can consider developers who have not committed to it as negatives? But they might still be owners? Possibly, if they are listed in CODEOWNERS but haven't committed yet. But we can exclude those from negatives if we know they are owners. But we don't know all owners; we only know from CODEOWNERS. So for a code artifact, we have a set of positive owners (from CODEOWNERS). We can sample negative developers from the set of all developers who have committed to the repo but are not in that positive set. That's a reasonable negative set: they are active developers but not owners of that file. Then we can train a model to distinguish between (code, owner) and (code, non-owner) pairs.

    But note: the model should also be able to predict owners who have never committed to that file? That's a challenge because they have no commit history with that file. But if they are listed in CODEOWNERS, they might be owners without having committed yet. In that case, we cannot rely on commit history between them and the file. But we might capture that through the graph: they might be connected to other files they own, and those files might be connected to the target file via co-change. So if we include co-change edges between code artifacts, then an owner of a related file might be recommended for the target file. So we should include co-change edges.

    Thus, we should add edges between code artifacts that are frequently changed together. This will help propagate ownership signals.

    Graph Construction Details:

    We have two types of nodes: developers (D) and code artifacts (C). Edges:

    1. Commit edges: between d in D and c in C if d has committed to c. We can weight by number of commits, or recentness. For simplicity, we can make it binary (at least one commit) or weighted by commit count.
    2. Co-change edges: between c1 and c2 if they are often changed together in the same commit (i.e., in the same commit, multiple files are changed; we can connect all pairs of files changed in the same commit, or use a threshold). This is an undirected edge between code artifacts.

    We do not have direct edges between developers (unless we add collaboration edges? maybe not needed).

    So the graph is heterogeneous? Actually, we have two node types but edges are either between D and C (commit) or between C and C (co-change). So it's a heterogeneous graph with two node types and two edge types. But we can treat it as homogeneous by projecting? Better to use a heterogeneous GNN or Metapath2Vec.

    For simplicity, we can run random walks that respect meta-paths. For example, we can have meta-paths:

    • D-C-D (developer-code-developer)
    • C-D-C (code-developer-code)
    • C-C (code-code)

    We can use Metapath2Vec to generate embeddings that capture these transitions.

    Alternatively, we can use a unified graph where we just have one type of node and encode the type as a feature? But then we lose type information. It's better to keep types.

    Given the complexity, maybe we start with a simpler homogeneous graph: we treat developers and code artifacts as the same type? That would be weird because they are different. But we could create a bipartite graph and then use Node2Vec with a bias to alternate between the two partitions? Node2Vec can be adapted for bipartite graphs by setting the return parameter and in-out parameter to encourage alternating? Actually, Node2Vec can handle bipartite graphs if we set the parameters appropriately (p and q). But it will produce embeddings in a shared space. That might be okay: then we can compute similarity between any two nodes, but we only care about code-developer pairs. That could work.

    We'll need to decide.

    Labeling for Training:

    We have ownership labels from CODEOWNERS. For each code artifact, we have a set of owner developers (could be multiple). We'll create positive pairs: (code, owner). For negative pairs, we can sample developers from the set of all developers who have ever committed to the repository but are not owners of that code artifact. But we need to be careful: we don't want to sample developers who are owners of other files? That's okay; they are still negative for this file. But we might also want to include developers who have never committed to the repo? That would be obviously negative, but they are not in the graph? Actually, we only have developer nodes for those who have at least one commit in the repo (since we build the graph from commit history). So our developer set is limited to contributors. That's fine because owners are usually contributors.

    So for each positive pair, we can sample k negative developers (randomly from the developer set excluding the positives). That gives us a training set of pairs with binary labels.

    Then we can train a model that takes the embedding of the code node and the embedding of the developer node (concatenated or dot product) and outputs a score. We can use a logistic regression or a neural network.

    But we need to get the embeddings first. We can do:

    • Step 1: Use Node2Vec (or Metapath2Vec) on the entire graph (with commit and co-change edges) to get initial embeddings for all nodes. These embeddings capture the graph structure (collaboration patterns, code similarity).
    • Step 2: For each (code, developer) pair in our labeled dataset, we take the corresponding embeddings and train a classifier to predict ownership.

    But this is a two-step process and might not be optimal because the initial embeddings are not optimized for the ownership task. We could instead use a GNN that learns embeddings directly with the supervision signal. That would be more integrated.

    So we can use a GNN (like GraphSAGE) that can handle heterogeneous graph? There are methods for homogeneous graphs. We can make the graph homogeneous by adding node features that indicate type (e.g., a one-hot feature for node type: developer or code). Then we can run GraphSAGE. But we have two edge types: commit and co-change. We could treat them as the same? Or we can have separate edge weights. For GraphSAGE, we can aggregate from neighbors regardless of edge type, but we might want to distinguish. Alternatively, we can use a relational GraphSAGE that has different weight matrices per edge type. That would be better.

    Given the complexity and the need for a practical solution, maybe we start with a simpler approach: use only commit edges (bipartite graph) and Node2Vec embeddings, then train a logistic regression on top. That might be a good baseline.

    But we also want to incorporate code content. So we need to provide initial node features for code nodes. We can use a pre-trained code embedding model to get a feature vector for each code artifact (file). For example, use CodeBERT to get a vector for the entire file (by averaging token embeddings). But that might be expensive for large repos. Alternatively, we can use a simpler bag-of-words or TF-IDF on the code text (after tokenizing). But that might not capture semantics well.

    We could also use the graph structure to propagate information from code to code via co-change edges, and from developers to code via commit edges. If we have good initial features for code (like code embeddings), then the GNN can combine that with structural information.

    So a plan:

    • For each code artifact (file), compute a feature vector using a language-agnostic code embedding model (e.g., CodeBERT, or a simpler model like Word2Vec trained on code tokens across languages). We'll need to decide on a model. Since we want language-agnostic, we should use a model trained on multiple languages. CodeBERT is a good choice, but it's large. Alternatively, we can use a smaller model like fastText embeddings trained on code tokens from multiple languages. Or we can use a uniform representation: e.g., count of language-agnostic tokens (like keywords, operators) but that might lose information.

    • For developers, we can start with no features (or simple features like total commits, tenure). But we can also have embeddings from the graph structure.

    Now, for the model:

    We can use a GraphSAGE-like model that takes initial node features (for both code and developer nodes) and learns to predict ownership. We'll treat it as a node classification on code nodes? But we want to output a set of developers. That's multi-label classification on code nodes with a large label space (all developers). That's not feasible if we have many developers. Instead, we can treat it as a link prediction: we want to predict edges (code, developer) with label 1 for ownership. So we can use a GNN that produces embeddings for all nodes, and then we compute scores for (code, developer) pairs using a decoder (like dot product). We then have a binary classification loss on the positive and negative pairs.

    This is a common approach for link prediction with GNNs. We can use a framework like PyTorch Geometric with a GNN and a LinkPrediction task.

    Steps:

    1. Build a graph with nodes and edges (commit edges and co-change edges). We'll have two edge types? We can combine them into one graph with multiple edge types, but for simplicity we can just use commit edges and hope that co-change is captured through multi-hop? Actually, co-change edges are direct connections between code nodes, which are important. So we should include them. We can treat the graph as having two edge types, and use a relational GNN that aggregates separately for each edge type and then combines. Or we can just add all edges as one type? That would mix commit and co-change, but that's okay because they both indicate relationships. However, the semantics are different: commit edges connect code to developer, co-change connect code to code. But if we treat all as undirected edges between any nodes, then a developer node would be connected to code via commit, and code to code via co-change. That's a homogeneous graph. That's fine. We just need to have a way to differentiate node types? We can include node type as a feature (one-hot). So we can create a graph where:
      • Node features for code: the code embedding vector (e.g., 768-dim from CodeBERT).
      • Node features for developer: maybe a one-hot for developer ID? That would be high-dimensional and sparse. Better to use a learned embedding? Or we can start with a constant vector for developers and let the GNN learn. Alternatively, we can provide developer features like the number of commits, etc. But we want to be language-agnostic, so developer features are not language-specific. We can use a one-hot for developer and then use an embedding layer to get a dense vector. But the number of developers can be large. However, we are training on multiple repositories, so we might have thousands of developers. That's manageable with an embedding layer.

    So we can have:

    • For code nodes: initial features = precomputed code embedding (fixed or trainable? We can freeze or fine-tune).
    • For developer nodes: initial features = learnable embedding (size, say, 128). Or we can use aggregated statistics (like total commits, average commit size, etc.) but that might be repository-specific. If we want to generalize across repos, we might not want to use repo-specific stats. Actually, we are training on multiple repos, so we can use developer features that are global (like their overall experience across all repos? but we don't have that at inference for a new repo). So maybe just an ID embedding is better, but then we need to handle new developers at inference (cold start). That's a problem: if a new developer appears in the target repo, we have no embedding for them. But we can initialize new developer nodes with a default vector (e.g., average of existing) or use their code contributions? But they haven't contributed yet? Actually, if they are new, they might not have any commits yet. But we want to predict ownership for code artifacts; the owners are usually existing developers. So maybe we only consider developers who have at least one commit in the repo. So at inference, we have the commit history up to now, so we know who has contributed. So we can build the graph with all developers who have at least one commit. So cold start is not an issue if we restrict to existing developers. But what about a new developer who is about to be assigned as owner? That's the prediction target: we want to infer ownership, possibly for developers who have not yet committed to that file. But if they haven't committed to the repo at all, they wouldn't be in the graph. So we might want to include all developers in the organization? That's not feasible. So we limit to current contributors. That's acceptable.

    Thus, we can use an embedding for each developer ID, and at inference we initialize new developers (who have no commits yet) with a zero vector or average? But they won't have any commits, so they won't be connected to any code. That means they will have no influence from the graph. But they could still be owners if they are assigned later. But if they have no commits, we have no data to associate them with code. So it's reasonable that we cannot predict them. So we only predict among developers who have at least one commit.

    So we can have:

    • Node features:
      • Code nodes: a fixed vector from a pre-trained code model (e.g., CodeBERT). We can optionally fine-tune the code model? That would be expensive. We can keep it fixed and let the GNN adapt through message passing.
      • Developer nodes: a trainable embedding (lookup table) indexed by developer ID. This embedding is learned during training.

    We'll also need to handle multiple repositories: we train on many repos, and at inference we have a new repo. We want to use the model trained on many repos and apply it to a new repo. That means we need to build the graph for the new repo and run the GNN to get embeddings for the new nodes? But the GNN parameters are fixed from training. For new developer nodes, we have their embedding (from the lookup table) but we haven't seen them during training? That's a problem: the lookup table is only for developers seen during training. For new developers in the target repo, we need an out-of-vocabulary (OOV) handling. We can initialize OOV developer nodes with a special vector or with the average of all developer embeddings. But then during GNN message passing, they will get information from their neighbors (code nodes). That might be okay. But we need to ensure that the model can handle unseen developer IDs. Alternatively, we can not use an embedding lookup for developers, but instead use features derived from their activity (like number of commits, tenure, etc.) that can be computed for any developer. That would be more general. But those features are repository-specific. At inference, we can compute them for the new repo. That's okay because we have the commit history. So we can use:

    • Developer features:
      • Total number of commits in the repo.
      • Number of different files touched.
      • Tenure (time since first commit).
      • Average commit size (lines changed).
      • etc.

    These are numeric and can be normalized. They are language-agnostic.

    Similarly, we could have code features beyond the code embedding? The code embedding already captures content. So we can use the code embedding as the main feature, and optionally add some metadata (like file extension, LOC) but those might be correlated with language.

    We want to be language-agnostic, so we should avoid language-specific features. Code embedding from a multilingual model is good.

    So plan:

    • Precompute for each code artifact (file) a vector using a pre-trained multilingual code model (e.g., CodeBERT). We'll call this code_embedding.
    • For each developer, compute features: dev_features = [total_commits, num_files, tenure, avg_commit_size] (normalized per repository? or across training? We'll decide).
    • Build graph with nodes: code nodes (with code_embedding as feature) and developer nodes (with dev_features as feature). Edges: commit edges (undirected) and co-change edges between code nodes.

    Then we train a GNN (like GraphSAGE) that takes this graph and node features, and for each code node, we want to predict which developer nodes are owners. We can do this by:

    • Using the GNN to get embeddings for all nodes.
    • For each code node, compute scores with all developer nodes (or a sampled set) using a decoder (e.g., dot product or MLP).
    • Use binary cross-entropy loss on (code, developer) pairs with ownership labels.

    But we have many developers; we cannot compute scores with all for each code during training. So we use negative sampling: for each positive (code, owner) pair, we sample a few negative developers (who are not owners of that code). Then we compute loss for these pairs.

    Alternatively, we can treat it as a multi-label classification where the label space is all developers, but that's huge. So negative sampling is standard for link prediction.

    So training procedure:

    For each epoch:

    • Sample a batch of positive (code, owner) pairs from the training set.
    • For each positive pair, sample k negative developers (random from all developers in the repo, excluding the positives for that code).
    • Compute the GNN embeddings for the involved nodes (code and developers). We need to run the GNN on the whole graph? That can be expensive if the graph is large (many files and developers). We can do full-batch if the repo is not too huge, or use mini-batch sampling of nodes. Since we have multiple repos, each repo might have thousands of files and hundreds of developers. That's manageable for full-batch on a GPU? Possibly not for very large repos. We can use neighbor sampling (like GraphSAGE) to generate mini-batches.

    We'll need to implement a data loader that samples (code, developer) pairs and then fetches the computational subgraph for those nodes (using multi-hop neighbors). We can use PyTorch Geometric's NeighborLoader.

    We'll have a heterogeneous graph? Actually, we have two node types and two edge types. PyTorch Geometric supports heterogeneous graphs. We can define:

    • node types: 'code' and 'dev'
    • edge types: ('dev','committed_to','code'), ('code','cochanged_with','code')

    Then we can use a HeteroGraphSAGE or a custom GNN that handles multiple edge types.

    Simpler: we can convert to homogeneous by adding a node type feature, but then we need to handle edge types separately in the aggregation. It's easier to use a heterogeneous GNN.

    Alternatively, we can ignore co-change edges and just use commit edges? That might be a baseline. But co-change is valuable.

    Given the time, we'll design a system that can incorporate both.

    Summary of Training Data:

    • Sources: GitHub repos with commit history and CODEOWNERS (or we can use heuristic for repos without CODEOWNERS but then labels are noisy). We need a dataset of (repo, code_artifact, owner) triples.
    • Graph construction:
      • Nodes: For each code artifact (file) and each developer (with at least one commit).
      • Edges:
        • Commit edges: between developer and code if commit exists. We can weight by commit count or binary.
        • Co-change edges: between code files if they appear in the same commit (we can set a threshold on co-occurrence count to reduce noise).
      • Node features:
        • Code: precomputed embedding from a multilingual code model (e.g., CodeBERT base, 768-dim). We can precompute and store.
        • Developer: hand-crafted features (commits, files, tenure, avg lines) or an embedding lookup. We'll go with hand-crafted to avoid OOV issues and because they are interpretable and can be computed for any developer. But note: these features are repository-specific. At training, we compute them per repository. At inference, we compute them for the target repository. That's fine.
    • Labels: For each code artifact, the set of owner developers (from CODEOWNERS). We'll only include code artifacts that have at least one owner? Or we can also include negative examples for files without owners? Probably not, because we want to predict owners only for files that should have owners. But we might want to predict for any file. We can treat files without CODEOWNERS as having no owners? That might be noisy. Maybe we only train on files that have explicit CODEOWNERS? That limits data. But we can also use the heuristic (most frequent committer) for repos without CODEOWNERS, but that's a different distribution. We need to decide.

    We could use two training regimes:

    1. Supervised: Only use repos with CODEOWNERS, and use CODEOWNERS as ground truth.
    2. Weak-supervised: Use all repos, and for each file, the owner is the developer with the most commits (or the one who created it). That's noisy but large.

    We might combine both: use CODEOWNERS as high-confidence positives, and for repos without CODEOWNERS, use the heuristic as positives. But then we have two different label qualities. We can weight the loss accordingly.

    Given the complexity, let's start with a supervised dataset using only repos with CODEOWNERS. That provides clean labels. We can augment with more repos later.

    Bias Mitigation:

    Potential biases:

    • Temporal bias: Recent commits are weighted more? If we use commit count, senior developers with long history may dominate. But ownership should be current. We might want to weight recent commits more. Or we can use only commits in the last N months.
    • Popularity bias: Files that are changed often may have many committers, making ownership ambiguous. But owners might be a small subset. Our model might learn to predict the most frequent committers, which may not be the actual owners if the file is widely touched.
    • Team bias: If a team works on a set of files, the model might assign ownership to the whole team even for files that have specific owners.
    • Language bias: If we train on mostly Python repos, the model may not work well for Java. We need diverse languages.

    Mitigation strategies:

    • Data balancing: Ensure training data includes diverse languages, team sizes, file types, etc.
    • Feature normalization: When computing developer features (like total commits), we can normalize per repository (e.g., rank within repo) to avoid absolute count bias.
    • Temporal splitting: When evaluating, use time-based splits (train on older data, test on newer) to simulate real-world scenario and avoid data leakage.
    • Regularization: Use dropout, weight decay.
    • Adversarial debiasing: We could try to make the model's predictions independent of certain sensitive attributes (like developer tenure, gender inferred from name? but that's problematic). We might not have sensitive attributes. But we can consider "seniority" as a bias we want to avoid? Actually, senior developers might be more likely to be owners; that may be legitimate. But we don't want to always favor seniority if a junior is the actual owner. So we need to be careful.

    We can also incorporate fairness constraints: e.g., ensure that the model's predictions are not correlated with developer's tenure beyond what is justified. But we don't have ground truth for fairness.

    Better: during training, we can sample negative developers in a way that balances certain attributes. For each code artifact, we have a set of positive owners. We sample negatives such that the distribution of developer tenure (or commit count) is similar to the positives? That would force the model to look beyond simple features.

    Alternatively, we can use a loss that includes a fairness regularizer: we want the predicted probability for a developer to be independent of their tenure, for example. But that's complex.

    Simpler: we can evaluate the model for bias by measuring correlation between predicted ownership and developer seniority (tenure). If there is high correlation, we might want to adjust. But maybe it's okay if senior developers are indeed more likely to be owners.

    We'll discuss bias mitigation in the validation section.

    Validation:

    We need to evaluate the model's ability to predict owners for code artifacts.

    Metrics:

    • Precision@k: For each code artifact, we rank all developers by predicted score. We look at the top k (e.g., k=1, 3, 5) and see if the true owner(s) are in that list. Since there can be multiple owners, we can compute precision at k (fraction of top-k that are true owners) and recall at k (coverage of true owners). Also, we can compute Mean Average Precision (MAP) or NDCG if we have graded relevance? But we have binary ownership.

    • Hit Rate: For each code artifact, does the top-1 predicted developer match the primary owner? But there may be multiple owners, so we might want to see if the set of top-k covers all owners. We can use set-based metrics like F1 for the top-k set vs. true set? That's more complex.

    Typically, ownership inference is like a recommendation: we want to retrieve the right owners. So we can use:

    • Recall@k: For each file, fraction of true owners that appear in the top-k predictions. Then average over files.
    • Precision@k: For each file, number of true owners in top-k divided by k. Average over files.
    • F1@k: harmonic mean.

    Also, we can compute the Mean Reciprocal Rank (MRR): for each file, take the reciprocal of the rank of the first true owner (if multiple, we can take the best rank). Then average.

    But note: our model produces a score for every (file, developer) pair. We rank developers per file.

    We also care about the coverage: what percentage of files have at least one true owner in the top-k? That's like hit rate.

    We should also evaluate on a per-repository basis and aggregate.

    Validation Strategy:

    • Train/validation/test split by time: Since code ownership changes over time, we should split by time. For each repository, we can use the first 80% of the commit history (by time) to build the graph and train, and the last 20% to test? But ownership labels (CODEOWNERS) are static? CODEOWNERS files change over time as well. So we can use a time-based split: for a given cutoff date, we use all commits before that date to build the graph and also consider the CODEOWNERS file at that time? But CODEOWNERS might not be versioned? Usually it is. So we can get the CODEOWNERS at the cutoff date. Then we test on files that were modified after the cutoff? But we want to predict ownership for any file, not just those modified after. Actually, we want to predict current ownership. So we can use the entire history up to now to build the graph, and then predict for all files using the current CODEOWNERS as ground truth. But that would be transductive: we have the graph with all nodes, and we predict on the same graph. That's not a good test because we might overfit to the graph structure. We need to simulate future prediction: we want to predict ownership for files that will be modified in the future, or for new files? But our model doesn't handle new files easily because we need code embeddings for new files. But we can compute that on the fly.

    Better: we split repositories into train/val/test. That is, we train on a set of repositories, validate on a held-out set, and test on another held-out set. This tests generalization across repositories. But we might have language bias if train and test have different language distributions. We should ensure language diversity in splits.

    We can do:

    • Split repositories randomly into 70% train, 15% val, 15% test. But we must ensure no overlap of developers? Not necessary because developers can be in multiple repos. But we want to test on unseen repositories, so it's fine if developers overlap? Actually, if a developer appears in both train and test repos, the model might have learned their embedding from train and apply to test. That's okay because we want to predict for any developer. But we need to handle OOV developers: if a developer only appears in test, we need to have a way to initialize them. In our design, we have developer features computed per repository, and we have an embedding lookup? If we use hand-crafted features, we don't need a lookup; we can compute features for any developer in the test repo. So we can have new developers in test. That's fine. But if we use an embedding lookup, we need to handle OOV. So hand-crafted features are better for OOV.

    So we'll use hand-crafted developer features.

    Thus, we can train on a set of repos, and at test time, for a new repo, we compute the graph and features for that repo, run the GNN (with the same model parameters) to get predictions. That's transductive? Actually, we run the GNN on the entire graph of the test repo to get embeddings for all nodes, then we compute scores for all (code, developer) pairs. That's fine. But the GNN parameters are trained on train repos. So we need to ensure that the GNN can generalize to new graphs. That's typical for GNNs: they are trained on a set of graphs and applied to new ones. But our training might have multiple graphs (repos). We can train in a multi-graph setting: each epoch, we sample a repository, build its graph, run GNN, compute loss on that graph's ownership pairs. That's like training on multiple independent graphs. That should work.

    So we'll do:

    • Training: For each epoch, randomly sample a repository from the training set. Build its graph (with commit and co-change edges) and compute node features (code embeddings, developer features). Then run the GNN to get node embeddings. Sample a batch of (code, developer) pairs with labels (from CODEOWNERS) and negative samples. Compute loss (binary cross-entropy on scores). Backprop.

    • Validation: Periodically evaluate on the validation set repositories (maybe one by one or average).

    • Test: After training, evaluate on test set repos.

    We need to be careful about data leakage: we must not use any information from the test repo during training. That includes the code embeddings? The code embeddings are derived from the code content, which is repo-specific. That's fine because at test time we'll compute code embeddings for the test repo using the same pre-trained model. So we are using a fixed pre-trained model for code embeddings. That model is trained on a large corpus of code (including possibly the test repos? but that's okay because it's a general model). But if the pre-trained model has seen the test repo's code during its pre-training, that's not a big issue because it's just a feature extractor. However, to be rigorous, we could fine-tune the code model on the training repos? That would leak test info if we use test code. But we can keep it fixed.

    So we'll use a pre-trained CodeBERT (or similar) and freeze its parameters. We'll only train the GNN and maybe a decoder.

    Bias Mitigation During Training:

    • We can sample negative developers in a way that balances for developer seniority. For each positive (code, owner) pair, we can sample negatives such that the distribution of (e.g., total commits) is similar to the positive owner's stats. That's like hard negative mining? Actually, we want to avoid the model learning to just pick the most prolific developers. So we can sample negatives that are also prolific, making the task harder. But that might not remove bias; it might force the model to learn more nuanced signals.

    We can also add a regularization term that discourages the model from relying too much on developer seniority features. For example, we can add a loss that makes the model's prediction independent of the developer's tenure (via adversarial training). That is complex.

    Alternatively, we can evaluate the bias and if found, we can adjust by re-weighting examples or using a fairness-aware loss.

    We'll include in the validation metrics an analysis of bias: e.g., compute the correlation between predicted scores and developer tenure (or commit count). If high, we might need to mitigate.

    We can also include fairness as a validation metric: e.g., equal opportunity across groups (if we have group labels, like gender, but we may not). So maybe we skip that.

    Integration Plan:

    We want to integrate this engine into a development workflow. Possible use cases:

    • CODEOWNERS suggestion: When a new file is added, or when a file lacks CODEOWNERS, suggest likely owners.
    • Code review assignment: When a pull request modifies certain files, suggest reviewers based on predicted owners.
    • Onboarding: Help new developers find relevant code owners.

    Integration steps:

    1. Deployment as a service: The engine can be packaged as a REST API or a command-line tool. It takes as input a code artifact (file) and the repository context (commit history, current CODEOWNERS) and returns a ranked list of recommended owners.

    2. Data pipeline: For a given repository, we need to construct the graph from the commit history and compute code embeddings. This could be done on-demand (for small repos) or precomputed periodically (for large repos). We might need to update the graph as new commits come in.

    3. Inference: For a target file (maybe at creation time or when CODEOWNERS is missing), we:

      • Compute its code embedding using the pre-trained model.
      • Build the graph for the repository up to the current state (or use a pre-built graph).
      • Run the GNN to get embeddings for all nodes (or just for the target file and all developers? We can compute embeddings for all nodes in one forward pass if we have the whole graph).
      • Compute similarity scores between the target file's embedding and all developer embeddings (or use a decoder MLP that takes pair of embeddings).
      • Return top-k developers.
    4. Integration points:

      • IDE plugin: As the developer creates a new file, the plugin can suggest owners.
      • CI/CD pipeline: When a PR is opened, the system can suggest reviewers based on files changed.
      • Git hook: Pre-commit hook to check if the file has an owner? Not sure.
      • Web dashboard: For project managers to see ownership distribution.
    5. Maintenance:

      • The model will need periodic retraining as new data arrives (new repos, new commit history). We can set up a pipeline that retrains every month or quarter.
      • The inference service needs to be updated with new model parameters.
      • For each repository, we need to keep the graph up-to-date. This can be done by listening to GitHub webhooks (push events) and incrementally updating the graph. But incremental GNN updates are non-trivial; we might recompute the graph periodically.
    6. Scalability:

      • For large repositories (e.g., millions of files), running GNN on the entire graph might be heavy. We can use sampling or restrict to relevant subgraph (e.g., only files in the same module). But for simplicity, we might assume repositories are not that huge. Many open-source repos have <100k files.
      • We can also precompute embeddings for all files and developers offline, and at inference just do a nearest neighbor search (if we use a simple dot product). That would be fast. But if we use a GNN that requires full graph propagation, we need to run it each time the graph changes. We could precompute embeddings periodically (e.g., daily) and store them. Then for a new file, we compute its embedding and find nearest neighbors among stored developer embeddings. But the GNN's embeddings for developers depend on the whole graph, so if we precompute, we need to also compute for all files? Actually, we can precompute embeddings for all existing files and developers. For a new file, we need to compute its embedding in the context of the graph. That requires running the GNN with the new node. That's not precomputed. We could run the GNN on the subgraph induced by the new file and its neighbors? But the GNN typically requires multiple hops, so we need to include neighbors of neighbors, etc. That could be a small subgraph if we limit. But the new file's embedding will be influenced by the entire graph through multiple hops? In GraphSAGE, we sample neighbors, so we can include a few hops. That can be done efficiently if we have the neighbor indices. So we can precompute embeddings for all existing nodes, and for a new file, we compute its embedding by aggregating from its neighbors (which are existing nodes with precomputed embeddings). That's possible if we use an inductive GNN like GraphSAGE. Indeed, GraphSAGE is inductive: after training, we can generate embeddings for new nodes by feeding their neighborhood features. So we can precompute embeddings for all existing nodes, and for a new file, we can compute its embedding on the fly by sampling its neighbors (commits from developers? But a new file has no commit history yet? Actually, when a new file is created, there is a commit that adds it. So at the time of creation, we know which developer committed it. That commit edge exists. Also, the file may be co-changed with other files in that commit. So we have initial edges. So we can compute its embedding by aggregating from its immediate neighbors (the developer who added it, and the other files in the same commit). Then we can get an embedding without running full graph propagation. That's efficient.

    So we can design the system to be inductive: we train a GraphSAGE model that can generate embeddings for new nodes given their features and neighbor features. Then we precompute embeddings for all existing nodes (files and developers) and store them (or compute on demand). For a new file, we compute its embedding using the trained model and its neighborhood, then find nearest developer neighbors (using the stored developer embeddings? But the developer embeddings might change if we add a new file? Actually, the developer embeddings are static after training? In GraphSAGE, the node embeddings are generated by the model; they are not updated when new nodes are added because the model parameters are fixed. The existing developer embeddings we computed earlier are based on the graph at training time. If we add a new file, we don't recompute embeddings for existing developers because that would require running GNN on the whole graph again. But the new file's embedding will be based on the existing developer embeddings (from the trained model). That's okay: the existing developer embeddings are fixed, and the new file's embedding is computed using those fixed embeddings. That's an inductive setting. So we can precompute all developer embeddings once (after training) and store them. Then for a new file, we compute its embedding by feeding its own features and the features of its neighbors (the developer who committed it, and co-changed files) into the model. The model will use the trained aggregators to combine neighbor embeddings (which we have stored for developers and for other files). For other files, we need their embeddings too. But if the co-changed files are existing, we have their embeddings stored. So we can do a one-hop or two-hop aggregation using stored embeddings. That is efficient.

    Thus, we can have a service that:

    • Periodically (or on demand) computes embeddings for all files and developers in a repo using the trained GNN (full graph). Stores them in a database.
    • When a new file is added, we:
      • Compute its code embedding (content).
      • Identify its immediate neighbors: the developer(s) who committed it (from the commit) and the other files in the same commit (co-change). Those neighbor nodes are existing (except maybe the developer is existing? The developer might be new to the repo? But if it's a new commit, the developer is an existing contributor? Possibly a new contributor? If new, we don't have a stored embedding for them. But we can compute their embedding on the fly? Actually, for a new developer (first commit), we need to generate an embedding. We could treat them as a new node with only the commit edge to this file and maybe other files? But they have no history. We could compute their embedding using their developer features (which we can compute from this single commit? Possibly: total commits=1, tenure = time since this commit, etc.). But in our GNN, the embedding for a developer node is generated from its features and its neighbors (code files it committed to). For a new developer, we have only this one file as neighbor. So we can compute their embedding by passing their features and the file's embedding through the GNN's aggregator for developer nodes. But we need the file's embedding first. But the file is new, we are computing its embedding. This is a chicken-egg. Actually, we need to compute both simultaneously? In GraphSAGE, we compute embeddings for all nodes in a batch, but for inductive setting we can compute new node embeddings given its neighborhood where neighbors are existing nodes with known embeddings. If the new node is a file, its neighbors are developers. Those developers might be existing (with stored embeddings) or new. If a developer is new (first commit), we don't have an embedding. But we could treat them as a new node and compute their embedding as well? But then we need their neighbors (the files they committed to). For a brand new developer, they have only this file as neighbor. So we could compute their embedding using their features and the file's embedding? That's circular. However, we can break the cycle by initializing new developer embeddings using their features only (without aggregation) for the first time, but that would be inconsistent with the GNN training which always aggregates. Alternatively, we can require that we only predict for files after the developer has at least one commit (so they are not new). But a new developer could be an owner of a new file they just added. That's likely: the person who creates a file is often an owner. So we want to handle that. In that case, we can treat the new developer as having an embedding that is just a function of their features (maybe a linear layer) without aggregation? But that's not what the GNN learned. However, during training, we always had developers with at least some commits? Not necessarily; some developers might have only one commit. That's okay. But when we precomputed developer embeddings, we computed them using the full graph, which includes all their commits. For a new developer with only one commit (to the new file), if we try to compute their embedding inductively, we would need the embedding of the new file to aggregate. But we don't have that yet. So we have a dependency.

    One solution: we can compute embeddings for both new file and new developer together by running a small GNN on the subgraph consisting of the new file and the new developer and maybe other existing nodes? But if the new developer only has this new file, then the subgraph is just two nodes connected. We could compute their embeddings by passing messages between them: the file's embedding depends on the developer's embedding, and vice versa. We could iterate a few steps until convergence. But that's complicated.

    Alternatively, we can avoid having new developers without any stored embedding by ensuring that we only predict after the developer has made at least one commit to some file in the repo. That's reasonable: if they just made their first commit, they are new and we might not have enough data to assign ownership beyond that file. But they are the obvious owner of that file. So we could have a fallback: if a developer is new (no precomputed embedding), we use their developer features directly to compute a score? Or we simply assign them as owner for the files they created? That's a rule-based fallback.

    Given the complexity, maybe we can simplify by not trying to predict for files that have just been created by a new developer? But that's exactly when we want prediction. We could handle it by: when a new file is added, we know the committing developer. We can simply recommend that developer as the owner (maybe along with others). That's a simple heuristic. Our model could improve upon that by also considering related files and other developers.

    So we can accept that for brand new developers, we might not have a good embedding, but we can still compute one using the GNN with their features and the file's features? Actually, in our GNN, the developer embedding is computed from its own features and aggregated messages from its neighboring code nodes. For a new developer with one neighbor (the new file), we would need the file's embedding to compute the message. But we can compute the file's embedding from its own features and aggregated messages from its neighboring developers (including the new one). So we have a mutual dependence. However, we can initialize both with their feature vectors and then perform a few rounds of message passing between them until convergence. That's doable if the subgraph is small. But in practice, the new file might also be co-changed with other existing files, so it will have other neighbors (existing files) that have precomputed embeddings. That breaks the cycle: the file's embedding can be computed from its co-change neighbors (existing files) and the developers (including the new one). But if we compute the file's embedding first, we need the developer embeddings. But if the new developer has no other files, we still need their embedding. We can initialize the new developer's embedding to their feature vector (or a learned vector) and then compute the file's embedding by aggregating from co-change neighbors (existing files) and the developer (using the initial developer embedding). Then we can recompute the developer embedding by aggregating from the file (using the newly computed file embedding). We can iterate a couple times. That's like belief propagation. It might work.

    But maybe we don't need to be that sophisticated: we can simply compute the file's embedding based only on its co-change neighbors (existing files) and ignore the new developer for the file embedding? But then the file's embedding won't reflect the new developer's involvement. However, the new developer is only connected to that file, so they won't get much signal anyway. For the purpose of recommending owners for that file, we want to consider that the new developer is likely an owner. So we need to include the edge between them. But we can compute the file's embedding by aggregating from its co-change neighbors (existing files) and from the new developer using a default embedding for the new developer (e.g., average of all developer embeddings). That would give some signal that the file is associated with a developer, but not specific. Then the score between the file and that new developer would be based on the dot product between the file's embedding (which includes a generic developer signal) and the new developer's specific embedding (which we compute from their features and the file). This is messy.

    Given the time, we might assume that we only predict for files that have been around for some time, so all developers who have touched them have stored embeddings. For brand new files, we could use a fallback: the committing developer is automatically suggested, plus maybe others based on co-change. But our model might not handle that perfectly. We can note this as a limitation.

    Alternatively, we can design the system to recompute embeddings for the whole repository periodically (e.g., daily). Then for a new file, we wait until the next recompute? That's not real-time. So we need an inductive method.

    Given the complexity of the integration, we can present a high-level plan without going too deep into the inductive inference details.

    Architecture:

    • Training pipeline:

      • Data collection from GitHub repos (with CODEOWNERS).
      • Graph construction per repo (commit edges, co-change edges).
      • Code embedding extraction using pre-trained model.
      • Developer feature computation.
      • GNN model (Heterogeneous GraphSAGE) training with link prediction loss.
      • Validation and testing.
      • Model export.
    • Inference service:

      • For a given repository, we have a background job that periodically (or on push) updates the graph and recomputes embeddings for all existing nodes (files and developers) using the trained GNN. These embeddings are stored in a database (e.g., vector DB for developers, and for files we might not need to store if we compute on the fly? But we need file embeddings for nearest neighbor search. We can store them as well).
      • When a request comes for a file (by path), we:
        • If the file is new (not in stored embeddings), we compute its code embedding, find its neighbors (commits and co-change) from the latest commit that introduced it, and then compute its embedding using the GNN's inductive method (by aggregating from neighbor embeddings, which may include existing files and developers). For any new developer neighbor, we compute their embedding on the fly as well? Or we fallback.
        • Once we have the file's embedding, we compute similarity (dot product) with all developer embeddings (stored) and return top-k.
      • To handle new developers: if a developer is not in the stored set, we compute their embedding using their features and their neighborhood (files they committed to). But if they are new, they may have only the new file. Again circular. We could compute their embedding by first computing the file's embedding (using other neighbors) and then aggregating from that file. That might work if the file has other co-change neighbors. If not, we could use just the developer's features as a proxy.

    This is getting complex. Maybe we can avoid storing developer embeddings and instead compute them on the fly for each request using the GNN on the current graph? But that would require running the GNN on the entire repo graph for each request, which is heavy. However, if we precompute and store, we need to update when the graph changes. So we need an incremental update mechanism.

    Given the scope, we can propose a simpler integration: batch prediction. The system runs offline periodically to generate a report of recommended owners for all files, or for files without owners. Then the report is used by maintainers. That's easier: we periodically rebuild the graph, run the GNN on the whole graph, compute file-developer scores, and output a mapping. That can be done daily. Then we can compare with current CODEOWNERS and suggest additions/changes.

    That is a practical integration: as a GitHub bot that opens PRs to update CODEOWNERS based on predictions. Or as a dashboard.

    So we can propose:

    • The engine runs as a scheduled job (e.g., nightly) for each repository of interest.
    • It fetches the latest code and commit history, builds the graph, computes embeddings, predicts owners for all files (or files without explicit owners), and compares with current CODEOWNERS.
    • It generates a report (e.g., HTML) or automatically creates a PR with suggested updates to CODEOWNERS.

    That avoids real-time serving complexities.

    Now, let's structure the answer.


    Proposed Solution Outline

    1. Training Data

    • Sources: Public GitHub repositories with permissive licenses, containing a CODEOWNERS file and a sufficient commit history (e.g., >100 commits, >5 contributors). We'll use the GitHub API to collect repos and clone them.
    • Extraction:
      • Parse CODEOWNERS to get ownership patterns (glob patterns) and map to specific files in the repo at a given commit (e.g., the HEAD of the training period). We'll expand patterns to actual file paths.
      • For each file, determine the set of owner developers (by email or GitHub ID) from CODEOWNERS. If a team is listed, we expand to its members via GitHub Teams API (if available) or ignore if not resolvable.
      • Extract commit history: for each commit, get the list of changed files and the author (and committer, but author is more relevant). We'll treat author as the developer.
      • Build commit edges: for each (developer, file) pair, count the number of commits (or binary if at least one).
      • Build co-change edges: for each commit, for every pair of files changed in that commit, increment a co-change count. After processing all commits, we keep edges with co-change count above a threshold (e.g., 5) to reduce noise.
    • Node Features:
      • Code artifacts: For each file, extract its content at a representative snapshot (e.g., the latest version before the ownership label time). Use a pre-trained multilingual code model (e.g., CodeBERT-base) to obtain a 768-dimensional embedding by averaging the last hidden states of tokens. Optionally, we can also add simple metadata (file extension as one-hot, LOC) but we'll start with just the code embedding.
      • Developers: For each developer (identified by email or GitHub ID), compute repository-specific features:
        • total_commits: number of commits authored.
        • num_files_touched: number of distinct files.
        • tenure: days since first commit in the repo (relative to the snapshot date).
        • avg_commit_size: average number of lines changed per commit.
        • Normalize these features per repository (e.g., min-max or standardization).
    • Graph Construction: For each repository, create a heterogeneous graph with:
      • Node types: code and dev.
      • Edge types:
        • (dev, COMMITTED_TO, code) with weight = commit count (or binary).
        • (code, COCHANGED_WITH, code) with weight = co-change count.
      • Node features as above.
    • Labeling: For each code node, the positive owners are the dev nodes that are listed in CODEOWNERS for that file (after pattern expansion). This yields a set of positive (code, dev) pairs. We'll treat it as a binary classification on edges (ownership). We'll also generate negative pairs by sampling developers from the same repository who are not owners of that file (including those who have committed and those who haven't? We'll sample from all developers in the repo to avoid trivial negatives). For each positive, we sample k negatives (e.g., k=5) to form a training instance.

    2. Model Architecture

    • GNN: We use a heterogeneous GraphSAGE (or RGCN) that can handle two edge types separately. We'll have separate aggregation functions for COMMITTED_TO and COCHANGED_WITH edges. The model will take node features and produce embeddings for all nodes in the graph.
      • For each node type, we have a transform layer to project features to a common dimension (e.g., 128).
      • For each edge type, we have an aggregator (e.g., mean, LSTM, or MLP) that combines neighbor embeddings.
      • We run multiple layers (e.g., 2-3) to capture multi-hop relationships.
    • Decoder: For a (code, dev) pair, we compute a score as the dot product of their embeddings (or pass through an MLP). We use binary cross-entropy loss.
    • Training:
      • We sample a repository, build its graph, and run the GNN to get embeddings for all nodes.
      • We then sample a batch of (code, dev) pairs with labels (positive and negative).
      • We compute the loss and backpropagate.
      • We use full-batch training if the graph fits in memory; otherwise, we use neighbor sampling (Mini-batch GraphSAGE).
      • We train across many repositories to learn general patterns.

    3. Validation and Evaluation

    • Splits: We split repositories into train/val/test (e.g., 70/15/15) ensuring language diversity. We also consider temporal split: for each repo, we use commits up to a cutoff date for training and the later period for test? But our labels (CODEOWNERS) are not temporal? They may change. To avoid leakage, we should use the CODEOWNERS file as of the cutoff date. But it's easier to split by repo. We'll do repo-level split.
    • Metrics:
      • Recall@k: For each code file, we rank all developers by predicted score. We compute the fraction of true owners that appear in the top-k. We then macro-average across files.
      • Precision@k: Fraction of top-k that are true owners.
      • Hit Rate@k: Whether at least one true owner is in top-k.
      • MRR: Mean reciprocal rank of the first true owner.
      • We also compute coverage: percentage of files for which we return any prediction (should be 100%).
    • Bias Analysis: We'll compute correlations between:
      • Predicted scores and developer seniority (tenure, total commits).
      • True ownership and seniority (to see if ground truth is biased).
      • We'll also stratify by file type (e.g., test files vs. core) and measure performance differences.
      • We'll report these to assess bias.

    4. Bias Mitigation Strategies

    • Data-level:
      • Ensure training data includes diverse languages, file types, and team sizes.
      • Balance the dataset: for each file, sample negative developers with similar seniority distribution as the positives (matched sampling). This forces the model to look beyond seniority.
      • Use time-based weighting: only consider commits in the last N months (e.g., 6) to reflect current ownership, reducing historical bias.
    • Model-level:
      • Include seniority features (tenure, commit count) in developer features? That might let the model use them, but we can then check if it's over-relying. Alternatively, we could exclude them to force the model to rely on graph structure and code content. But seniority might be a legitimate signal. We'll include them and then monitor.
      • Add adversarial debiasing: add a gradient reversal layer that tries to predict a sensitive attribute (e.g., tenure group) from the embeddings, and train the main model to maximize loss for that predictor. This encourages embeddings that are invariant to tenure. But we need to define groups; we could bin tenure.
      • Regularization: add L2 penalty on the magnitude of weights that interact with seniority features? Hard.
    • Post-processing:
      • If we observe systematic under-prediction for certain groups, we can adjust scores (e.g., equalize odds). But we need group memberships.

    We'll implement matched negative sampling as a primary mitigation.

    5. Integration Plan

    We propose two deployment modes:

    A. Batch Analysis (Offline)

    • Frequency: Nightly or weekly.
    • Process:
      1. For each monitored repository (list can be configured), fetch the latest code and full commit history.
      2. Build the graph, compute code embeddings, developer features.
      3. Run the trained GNN to obtain embeddings for all nodes.
      4. For every code file, compute scores with all developers, take top-k (e.g., k=5).
      5. Compare with current CODEOWNERS (if exists). Identify files where suggested owners differ.
      6. Generate a report (HTML or markdown) listing:
        • Files without CODEOWNERS with suggested owners.
        • Files with CODEOWNERS but where the suggested top owners don't match (maybe indicate discrepancies).
      7. Optionally, automatically open a pull request that updates the CODEOWNERS file with the top suggested owner for files without one, or adds comments.
    • Advantages: Simple, no real-time requirement, can handle large graphs offline.
    • Disadvantages: Not real-time; suggestions may be stale.

    B. Real-time Advisory (Online)

    • Service: Deploy a microservice (e.g., Flask/FastAPI) that can answer queries: given a repository and a file path, return top-k suggested owners.
    • Data Management:
      • For each repository, maintain a background job that updates the graph and precomputes embeddings for all existing files and developers periodically (e.g., every hour). Store embeddings in a vector database (e.g., FAISS) for fast nearest neighbor search.
      • When a new file is created (detected via webhook), we need to compute its embedding immediately. We'll use an inductive GNN: compute its code embedding, find its immediate neighbors (the committing developer and co-changed files from that commit). We'll fetch embeddings for those neighbors from the database (if they exist) and run a few steps of the GNN to obtain the new file's embedding. For a new developer (first commit), we compute their embedding from their features and the file's embedding (iteratively). This is complex but doable.
    • Integration points:
      • IDE plugin: As the developer creates a file, the plugin can call the service and show suggested owners.
      • Pull request bot: When a PR is opened, the bot analyzes the changed files and suggests reviewers based on predicted owners.
      • Git hook: Pre-commit hook could warn if the file has no owner? But that might be intrusive.
    • Advantages: Immediate feedback, can guide developers at the moment of creation.
    • Disadvantages: More engineering effort, need to handle stale embeddings and incremental updates.

    We recommend starting with Batch Analysis to validate the model and then move to Real-time if needed.

    6. Implementation Roadmap

    Phase 1: Data Collection and Model Development (Months 1-3)

    • Build data pipeline: scrape GitHub for repos with CODEOWNERS, clone, extract commits, CODEOWNERS.
    • Implement graph construction and feature extraction (CodeBERT inference, developer features).
    • Train baseline model (e.g., logistic regression on developer features only) for sanity check.
    • Implement GraphSAGE (heterogeneous) in PyTorch Geometric.
    • Train and validate on held-out repos.

    Phase 2: Bias Mitigation and Evaluation (Month 4)

    • Run bias analysis.
    • Implement matched negative sampling and adversarial debiasing (if needed).
    • Re-evaluate.

    Phase 3: Integration Prototype (Month 5)

    • Build batch analysis pipeline: for a set of repos, generate CODEOWNERS suggestions.
    • Create a simple web interface to view suggestions.
    • Optionally, build a GitHub bot that opens PRs with suggested updates (using GitHub API).

    Phase 4: Real-time Service (Month 6)

    • Design and implement inductive inference for new files.
    • Build a service with caching and vector DB.
    • Develop IDE plugin or PR bot integration.
    • Conduct user testing.

    Phase 5: Productionization and Monitoring (Ongoing)

    • Deploy service on cloud (e.g., AWS, GCP).
    • Set up monitoring for prediction latency, accuracy (via user feedback), and bias metrics.
    • Schedule periodic retraining with new data.

    7. Potential Challenges and Mitigations

    • Scalability: Large repos (>100k files) may cause memory issues. Solutions: use neighbor sampling, or split graph by subdirectory (modularization). Also, we can restrict to files in a certain path pattern? Not ideal.
    • Cold start for new repositories: For a repo with little history, the graph is sparse. We can fall back to simple heuristics (e.g., most frequent committer overall) or use language-specific models? We can also use a meta-learning approach, but that's advanced. We'll rely on our model being trained on many repos; even with few commits, it can still produce embeddings based on code content and limited structure.
    • Data quality: CODEOWNERS may be missing or outdated. We'll rely on it as ground truth, but it's not perfect. We can incorporate multiple signals (e.g., reviewer data) in future.
    • Model drift: Ownership patterns change. We need regular retraining.
    • Privacy: Some repos are private. We need to handle access; the service may need to run within the organization's infrastructure.

    8. Future Enhancements

    • Incorporate pull request review data to refine ownership.
    • Use temporal graph networks to capture ownership evolution.
    • Include developer collaboration network (who works with whom).
    • Fine-tune the code embedding model on the commit history (self-supervised) to be more ownership-aware.
    • Multi-lingual support: ensure code embedding model covers rare languages.

    This covers the requested aspects: training data, validation, bias mitigation, integration plan.

    Now we need to write it up in a clear, structured manner.

    Language-Agnostic Code Ownership Inference Engine: Design and Integration Plan

    This document outlines a comprehensive solution for inferring code ownership using graph embeddings. The engine is designed to be language-agnostic, scalable, and integrable into modern development workflows.


    1. Training Data

    1.1 Data Sources

    • GitHub public repositories with permissive licenses (e.g., MIT, Apache 2.0).
    • Inclusion criteria:
      • Contains a CODEOWNERS file (provides ground-truth ownership).
      • At least 100 commits and 5 contributors (ensures sufficient activity).
      • Supports multiple programming languages (to ensure language-agnosticism).
    • Data collected:
      • Full commit history (author, timestamp, changed files).
      • CODEOWNERS file contents (at repository HEAD or a specific snapshot).
      • Repository metadata (language distribution, team structure via GitHub Teams API).

    1.2 Graph Construction

    We model each repository as a heterogeneous graph with two node types and two edge types:

    Node TypeDescriptionFeatures
    codeSource code file (e.g., .py, .java)Content embedding: 768-dim vector from CodeBERT (multilingual model).Optional: file extension (one-hot), LOC, cyclomatic complexity (via lizard).
    devDeveloper (identified by email/GitHub ID)Engineered features (per-repository, normalized):- total_commits- num_files_touched- tenure (days since first commit)- avg_commit_size (avg lines changed)
    Edge TypeDescriptionWeight / Attributes
    (dev, COMMITTED_TO, code)Developer authored a commit touching file.Commit count (or binary if ≥1).
    (code, COCHANGED_WITH, code)Two files changed together in a commit.Co-change count (thresholded, e.g., ≥5).

    1.3 Labeling Ownership

    • From CODEOWNERS, expand patterns to actual file paths.
    • Resolve team mentions to individual members via GitHub Teams API (if available); otherwise skip.
    • For each file, create positive (code, dev) pairs for listed owners.
    • Negative sampling: For each positive pair, sample k (e.g., 5) developers from the same repository who are not owners of that file. Include both active committers and inactive ones to avoid trivial negatives.

    2. Model Architecture

    2.1 Graph Neural Network

    • Framework: PyTorch Geometric (heterogeneous support).
    • Architecture: Relational GraphSAGE (or RGCN) with separate aggregation for each edge type.
      • Layers: 2–3 layers, output embedding dimension 128.
      • Node transforms: Linear projections for both code and dev features to common dimension.
      • Aggregators:
        • COMMITTED_TO: Mean aggregation from neighboring developers.
        • COCHANGED_WITH: Mean/LSTM aggregation from neighboring code files.
      • Inductive capability: After training, the model can generate embeddings for new nodes given their features and neighborhood.

    2.2 Decoder & Loss

    • Score function: Dot product between code and dev embeddings (optionally passed through an MLP).
    • Loss: Binary cross-entropy with logits.
    • Training:
      • Full-batch for small repos (<10k nodes); else neighbor sampling (GraphSAGE mini-batch).
      • Optimizer: Adam (lr=0.001), weight decay=1e-5.
      • Multi-repo training: each epoch samples a random repository from the training set.

    3. Validation & Evaluation

    3.1 Data Splits

    • Repository-level split: 70% train, 15% validation, 15% test.
    • Stratification by primary language to ensure diversity.
    • Temporal consideration: Use the CODEOWNERS file and commit history as of a consistent snapshot date to avoid leakage.

    3.2 Metrics

    • Primary:
      • Recall@k: Fraction of true owners appearing in top-k recommendations (macro‑averaged over files).
      • Precision@k: Fraction of top-k that are true owners.
      • Hit Rate@k: % of files with ≥1 true owner in top-k.
      • MRR (Mean Reciprocal Rank): Average inverse rank of first true owner.
    • Secondary:
      • Coverage: % of files for which predictions are generated (should be ~100%).
      • Per-language performance: To detect language bias.

    3.3 Bias Analysis

    • Correlation checks:
      • Predicted scores vs. developer seniority (tenure, total commits).
      • True ownership vs. seniority (to understand ground‑truth bias).
    • Stratified evaluation:
      • By file type (e.g., test, core, config).
      • By team size (small vs. large projects).
    • Fairness metrics (if sensitive attributes available, e.g., inferred gender from names):
      • Equal opportunity, demographic parity.

    4. Bias Mitigation Strategies

    4.1 Data-Level

    • Matched negative sampling: For each positive (code, owner) pair, sample negatives whose seniority distribution (tenure, commit count) matches the owner’s. This prevents the model from learning simple “pick the most senior” rules.
    • Temporal weighting: Only consider commits from the last 6–12 months to reflect current ownership and reduce historical seniority bias.
    • Balanced datasets: Ensure training set includes diverse languages, file types, and repository sizes.

    4.2 Model-Level

    • Adversarial debiasing:
      • Add a gradient reversal layer that tries to predict a seniority group (e.g., quartiles of tenure) from the dev embeddings.
      • The main model is trained to maximize loss for this predictor, encouraging embeddings invariant to seniority.
    • Regularization: Apply L2 penalty on weights that heavily weight seniority‑related features (if included).

    4.3 Post‑Processing

    • If systematic bias is detected (e.g., low recall for junior developers), apply score adjustment per group to equalize opportunity.

    5. Integration Plan

    We propose two deployment modes, starting with the simpler Batch Analysis and progressing to Real‑time Advisory.

    5.1 Batch Analysis (Offline)

    • Frequency: Nightly or weekly.
    • Process:
      1. For each monitored repository:
        • Fetch latest code and full commit history.
        • Build graph, compute code embeddings (CodeBERT), developer features.
        • Run trained GNN to get embeddings for all nodes.
        • For every code file, compute scores with all developers; take top‑k (e.g., 5).
      2. Compare suggestions with existing CODEOWNERS.
      3. Generate a report (HTML/Markdown) highlighting:
        • Files without owners and suggested owners.
        • Discrepancies (files with owners but suggestion lists differ).
      4. Optionally, automatically open a PR that updates CODEOWNERS with top suggestions for ownerless files.
    • Advantages: Simple, no real‑time constraints, easy to monitor.
    • Disadvantages: Suggestions are not immediate; may lag behind changes.

    5.2 Real‑time Advisory (Online)

    • Service: Deploy a REST API (e.g., FastAPI) that answers: “Given a repository and file path, return top‑k owners.”
    • Data Management:
      • Precomputation: Background job updates each repository’s graph and precomputes embeddings for all existing nodes hourly. Store in a vector DB (e.g., FAISS) for fast nearest‑neighbor search.
      • Inductive inference for new files:
        • On file creation (detected via webhook), compute its CodeBERT embedding.
        • Identify immediate neighbors: committing developer(s) and co‑changed files from the same commit.
        • Fetch embeddings of existing neighbors from DB; for any new developer, compute their embedding on‑the‑fly using their features and the file’s embedding (iterative message passing for a few steps).
        • Compute file’s final embedding via a few GNN layers using neighbor embeddings.
        • Rank against all stored developer embeddings.
    • Integration points:
      • IDE plugin: As a developer creates/saves a file, show suggested owners.
      • PR bot: When a PR is opened, suggest reviewers based on changed files.
      • Git hook: Pre‑commit warning if file lacks an owner (optional).
    • Challenges: Incremental graph updates, handling new developers, ensuring low latency.

    6. Implementation Roadmap

    PhaseDurationMilestones
    1. Data & ModelMonths 1–3- Collect ~500 repos with CODEOWNERS.- Build graph pipeline and feature extractors.- Train baseline (logistic regression) and GraphSAGE.- Validate on held‑out repos.
    2. Bias & EvaluationMonth 4- Perform bias analysis.- Implement matched negative sampling and adversarial debiasing.- Re‑evaluate metrics.
    3. Batch PrototypeMonth 5- Build nightly batch pipeline.- Generate reports for a pilot set of repos.- Create simple web dashboard.
    4. Real‑time ServiceMonth 6- Design inductive inference for new files.- Implement API with caching.- Develop GitHub PR bot integration.- Conduct user testing.
    5. ProductionizationOngoing- Deploy on cloud (e.g., AWS ECS).- Set up monitoring (latency, accuracy via user feedback, bias dashboards).- Schedule periodic retraining (monthly).

    7. Challenges & Mitigations

    • Scalability: For huge repos (>100k files), use neighbor sampling or modular decomposition (by directory).
    • Cold start: For new repos with little history, fall back to simple heuristics (most frequent committer) until the graph grows.
    • Data quality: CODEOWNERS may be outdated or missing. Consider augmenting with reviewer data from PRs in future versions.
    • Model drift: Ownership changes over time; schedule regular retraining with fresh data.
    • Privacy: For private repos, run the service inside the organization’s network; use fine‑grained access control.

    8. Future Enhancements

    • Incorporate pull request review data to refine ownership signals.
    • Use temporal graph networks to model ownership evolution.
    • Add developer collaboration edges (e.g., who frequently reviews whose PRs).
    • Fine‑tune CodeBERT on commit messages (self‑supervised) to make embeddings more ownership‑aware.
    • Extend to function‑level granularity (currently file‑level).

    This design provides a robust, language‑agnostic framework for inferring code ownership, with clear pathways for validation, bias mitigation, and practical integration into developer tools.

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.