https://huggingface.co/RainbowPiBubbles/mini-gte-onnx/tree/main .gitattributes Safe 1.52…
- User
https://huggingface.co/RainbowPiBubbles/mini-gte-onnx/tree/main
.gitattributes
Safe 1.52 kB initial commit 15 minutes ago README.md
786 Bytes Create README.md 9 minutes ago config.json
Safe 520 Bytes Upload ONNX model and assets 12 minutes ago model.onnx
67.4 MB LFS Upload quantized ONNX model and tokenizer 3 minutes ago special_tokens_map.json
Safe 695 Bytes Upload ONNX model and assets 12 minutes ago tokenizer.json
Safe 711 kB Upload ONNX model and assets 12 minutes ago tokenizer_config.json
Safe 1.23 kB Upload ONNX model and assets 12 minutes ago vocab.txt
Safe
Can you make a single html file that lets me upload a json with several texts, calculate embeddings for each of them, and calculate a linear regression model(line of best fit) to predict based on the top 5 most correlated (r squared) variables from the embeddings to corresponding the number in the json. It only uses 5 parameters so it doesnt overfit. I should be able to edit the order Then, it should let me predict a ranking of my own text
Preferably like around 300 or less lines of code
Here is an example of what the json looks like: { "a": 1, "b": 0.8, "c": 0.6666666666666666, "d": 0, "e": 1, "f": 0.5714285714285714, "g": 0, "h": 0 }
- Assistant
Here's a compact HTML file that accomplishes what you want using the ONNX model from HuggingFace, ONNX Runtime for inference, and basic linear regression. This solution uses the tokenizer and model from the repository you linked, handles file uploads, calculates embeddings, performs feature selection based on correlation, fits a linear regression model, and allows predictions.
<!DOCTYPE html> <html lang="en"> <head> <meta charset="UTF-8"> <title>Text Embedding Regression</title> <script src="https://cdn.jsdelivr.net/npm/onnxjs@0.1.8/dist/onnx.min.js"></script> <script src="https://cdn.jsdelivr.net/npm/axios/dist/axios.min.js"></script> </head> <body> <h1>Text Embedding Regression</h1> <input type="file" id="jsonFile" accept=".json"> <button onclick="processFile()">Process</button> <div id="status"></div> <h2>Top 5 Features (Drag to reorder)</h2> <ul id="features" style="list-style-type: none;"></ul> <button onclick="trainModel()">Train Model</button> <h2>Predict New Text</h2> <textarea id="newText" rows="4" cols="50"></textarea><br> <button onclick="predict()">Predict</button> <div id="prediction"></div> <script> const session = new onnx.InferenceSession(); const tokenizerUrl = 'https://huggingface.co/RainbowPiBubbles/mini-gte-onnx/resolve/main/tokenizer.json?download=true'; const modelUrl = 'https://huggingface.co/RainbowPiBubbles/mini-gte-onnx/resolve/main/model.onnx?download=true'; let tokenizer, embeddings, targets, topFeatures, modelParams; // Load model and tokenizer async function init() { await session.loadModel(modelUrl); const tokenizerResp = await axios.get(tokenizerUrl); tokenizer = tokenizerResp.data; document.getElementById('status').innerText = 'Model and tokenizer loaded.'; } init(); // Tokenize text function tokenize(text) { const tokens = text.toLowerCase().split(/\s+/).map(word => tokenizer.vocab[word] || tokenizer.vocab['[UNK]']); return tokens.slice(0, 128).concat(Array(128 - Math.min(tokens.length, 128)).fill(0)); } // Get embeddings async function getEmbedding(text) { const inputIds = tokenize(text); const tensor = new onnx.Tensor(new Float32Array(inputIds), 'int64', [1, 128]); const outputMap = await session.run({ input: tensor }); return Array.from(outputMap.values().next().value.data); } // Process uploaded JSON async function processFile() { const file = document.getElementById('jsonFile').files[0]; const reader = new FileReader(); reader.onload = async function(e) { const data = JSON.parse(e.target.result); embeddings = []; targets = []; for (const [text, score] of Object.entries(data)) { const emb = await getEmbedding(text); embeddings.push(emb); targets.push(score); } selectTopFeatures(); }; reader.readAsText(file); } // Calculate correlation (R^2) function correlation(x, y) { const n = x.length; const meanX = x.reduce((a, b) => a + b, 0) / n; const meanY = y.reduce((a, b) => a + b, 0) / n; const num = x.reduce((a, b, i) => a + (b - meanX) * (y[i] - meanY), 0); const denX = Math.sqrt(x.reduce((a, b) => a + (b - meanX) ** 2, 0)); const denY = Math.sqrt(y.reduce((a, b) => a + (b - meanY) ** 2, 0)); const r = num / (denX * denY); return r ** 2; } // Select top 5 features based on correlation function selectTopFeatures() { const correlations = embeddings[0].map((_, i) => correlation(embeddings.map(e => e[i]), targets)); topFeatures = correlations.map((r2, i) => ({ index: i, r2 })) .sort((a, b) => b.r2 - a.r2) .slice(0, 5) .map(f => f.index); displayFeatures(); } // Display features with drag-and-drop function displayFeatures() { const ul = document.getElementById('features'); ul.innerHTML = ''; topFeatures.forEach((f, i) => { const li = document.createElement('li'); li.draggable = true; li.innerText = `Feature ${f} (R^2: ${correlation(embeddings.map(e => e[f]), targets).toFixed(4)})`; li.dataset.index = i; li.addEventListener('dragstart', e => e.dataTransfer.setData('text/plain', i)); li.addEventListener('dragover', e => e.preventDefault()); li.addEventListener('drop', e => { e.preventDefault(); const fromIdx = e.dataTransfer.getData('text'); const toIdx = li.dataset.index; [topFeatures[fromIdx], topFeatures[toIdx]] = [topFeatures[toIdx], topFeatures[fromIdx]]; displayFeatures(); }); ul.appendChild(li); }); } // Linear regression (Ordinary Least Squares) function trainModel() { const X = embeddings.map(e => topFeatures.map(f => e[f])); const y = targets; const Xt = X[0].map((_, i) => X.map(row => row[i])); // Transpose const XtX = Xt.map(row => X[0].map((_, j) => row.reduce((a, b, k) => a + b * X[k][j], 0))); const XtY = Xt.map(row => row.reduce((a, b, i) => a + b * y[i], 0)); modelParams = solveLinearSystem(XtX, XtY); // Solve XtX * beta = XtY document.getElementById('status').innerText = 'Model trained.'; } // Solve linear system (Gaussian elimination) function solveLinearSystem(A, b) { const n = A.length; const augmented = A.map((row, i) => [...row, b[i]]); for (let i = 0; i < n; i++) { let maxEl = Math.abs(augmented[i][i]), maxRow = i; for (let k = i + 1; k < n; k++) { if (Math.abs(augmented[k][i]) > maxEl) { maxEl = Math.abs(augmented[k][i]); maxRow = k; } } [augmented[i], augmented[maxRow]] = [augmented[maxRow], augmented[i]]; for (let k = i + 1; k < n; k++) { const c = -augmented[k][i] / augmented[i][i]; for (let j = i; j < n + 1; j++) { augmented[k][j] += c * augmented[i][j]; } } } const x = new Array(n).fill(0); for (let i = n - 1; i >= 0; i--) { x[i] = augmented[i][n] / augmented[i][i]; for (let k = i - 1; k >= 0; k--) { augmented[k][n] -= augmented[k][i] * x[i]; } } return x; } // Predict new text async function predict() { const text = document.getElementById('newText').value; const emb = await getEmbedding(text); const x = topFeatures.map(f => emb[f]); const pred = x.reduce((a, b, i) => a + b * modelParams[i], 0); document.getElementById('prediction').innerText = `Predicted score: ${pred.toFixed(4)}`; } </script> </body> </html>Explanation of the Code:
-
Dependencies:
- Uses
onnxjsfor ONNX model inference in the browser. - Uses
axiosto fetch the tokenizer JSON file.
- Uses
-
Initialization:
- Loads the ONNX model and tokenizer JSON from the HuggingFace repository.
-
Tokenization:
- A simple tokenization function is implemented to convert text into token IDs using the tokenizer's vocabulary. It pads/truncates to a fixed length of 128.
-
Embedding Generation:
- The
getEmbeddingfunction runs the ONNX model to generate embeddings for a given text.
- The
-
File Processing:
- Reads the uploaded JSON file, extracts text-score pairs, and generates embeddings for each text.
-
Feature Selection:
- Calculates the R² correlation between each embedding dimension and the target scores, then selects the top 5 most correlated features.
- Displays these features in a draggable list, allowing you to reorder them.
-
Linear Regression:
- Implements a basic ordinary least squares (OLS) linear regression using Gaussian elimination to solve the system of equations.
- Uses only the top 5 features to prevent overfitting.
-
Prediction:
- Allows you to input new text, generates its embedding, and predicts a score using the trained model.
-
UI:
- Provides a simple interface for uploading files, reordering features, training the model, and making predictions.
Notes:
- The code is kept under 300 lines by simplifying some aspects (e.g., tokenization, matrix operations).
- The JSON file should have text as keys and scores as values, matching your example.
- The model and tokenizer are loaded directly from HuggingFace, so an internet connection is required.
- The drag-and-drop feature allows reordering the top 5 features, which affects the regression model when retrained.
- Error handling is minimal to keep the code concise; you may want to add more robust error handling in a production environment.
To use this, save the code as an HTML file, open it in a browser, upload your JSON file, and follow the on-screen instructions.
-
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.