Grok 4.1 Fast· xai
Frontier Reasoning Traces
Build a file upload service: streaming, AV scanning, MIME validation, lifecycle, and…
- User
Build a file upload service: streaming, AV scanning, MIME validation, lifecycle, and signed URLs.
- Assistant
// server.js - Complete File Upload Service // Features: // - Streaming upload via Multer to temporary disk storage (/tmp) // - MIME type validation using magic bytes (file-type) // - AV scanning using node-clamscan (requires clamd daemon running) // - Streaming upload to AWS S3 after validation // - Automatic S3 lifecycle management (configured via AWS CLI example) // - Signed URLs for secure downloads (pre-signed GET URLs, expiring in 1 hour) // - Error handling, size limits, etc. // // Prerequisites: // 1. Node.js 18+ // 2. AWS account with S3 bucket (e.g., 'my-upload-bucket') - IAM user with s3:PutObject, s3:GetObject // 3. Install ClamAV: Ubuntu `apt install clamav clamav-daemon`, start `systemctl start clamav-daemon`, `freshclam` // 4. npm install express multer @aws-sdk/client-s3 clamscan file-type uuid // 5. Set env: AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_REGION=us-east-1, UPLOAD_BUCKET=my-upload-bucket // 6. Run: node server.js // 7. Test: curl -F "file=@test.jpg" http://localhost:3000/upload?allowedMimes=image/jpeg,image/png&maxSize=10MB // 9. Lifecycle: See setup below# Setup S3 Lifecycle Policy (delete files > 30 days old) aws s3api put-bucket-lifecycle-configuration \ --bucket $UPLOAD_BUCKET \ --lifecycle-configuration '{ "Rules": [ { "ID": "DeleteOldUploads", "Status": "Enabled", "Filter": { "Prefix": "" }, "Expiration": { "Days": 30 } } ] }'const express = require('express'); const multer = require('multer'); const path = require('path'); const fs = require('fs'); const { v4: uuidv4 } = require('uuid'); const { S3Client, PutObjectCommand, GetObjectCommand } = require('@aws-sdk/client-s3'); const { getSignedUrl } = require('@aws-sdk/s3-request-presigner'); const ClamScan = require('clamscan'); const { fromFile: fileTypeFromFile } = require('file-type'); const app = express(); const port = process.env.PORT || 3000; // Config const bucketName = process.env.UPLOAD_BUCKET || 'my-upload-bucket'; const region = process.env.AWS_REGION || 'us-east-1'; const maxFileSize = parseInt(process.env.MAX_FILE_SIZE_MB || '100') * 1024 * 1024; // Default 100MB const signedUrlExpiry = '1h'; // 1 hour const s3Client = new S3Client({ region }); // Temp storage for streaming (disk-based, auto-cleanup) const upload = multer({ storage: multer.diskStorage({ destination: (req, file, cb) => cb(null, '/tmp'), filename: (req, file, cb) => cb(null, `${uuidv4()}-${file.originalname}`), }), limits: { fileSize: maxFileSize }, fileFilter: (req, file, cb) => { // Basic Content-Type check (client-side) if (!file.mimetype.startsWith('image/') && !file.mimetype.startsWith('application/') && !file.mimetype.startsWith('text/')) { return cb(new Error('Unsupported MIME type'), false); } cb(null, true); }, }); // Init ClamScan (connects to local clamd) let clamscan; (async () => { clamscan = await new ClamScan({ clamdscan: { socket: false, // Use TCP host: '127.0.0.1', port: 3310, }, clamdscanConfig: { noSummary: true }, }).init(); console.log('ClamScan initialized'); })(); // Helper: Validate MIME with magic bytes async function validateMime(filePath, allowedMimes) { const type = await fileTypeFromFile(filePath); const detectedMime = type?.mime; if (!detectedMime || !allowedMimes.includes(detectedMime)) { throw new Error(`Invalid MIME type: ${detectedMime}. Allowed: ${allowedMimes.join(', ')}`); } return detectedMime; } // Helper: Stream file to S3 async function uploadToS3(filePath, originalName, detectedMime) { const key = `uploads/${uuidv4()}-${originalName}`; const fileStream = fs.createReadStream(filePath); const params = { Bucket: bucketName, Key: key, Body: fileStream, ContentType: detectedMime, }; await s3Client.send(new PutObjectCommand(params)); return key; } // Helper: Generate signed download URL async function generateSignedUrl(key) { const command = new GetObjectCommand({ Bucket: bucketName, Key: key, }); return await getSignedUrl(s3Client, command, { expiresIn: 3600 }); // 1 hour } // Upload endpoint app.post('/upload', upload.single('file'), async (req, res) => { try { if (!req.file) { return res.status(400).json({ error: 'No file uploaded' }); } const filePath = req.file.path; const allowedMimes = (req.query.allowedMimes || 'image/jpeg,image/png,application/pdf').split(','); // 1. MIME validation (magic bytes) const detectedMime = await validateMime(filePath, allowedMimes); console.log(`MIME validated: ${detectedMime}`); // 2. AV scanning const { isInfected } = await clamscan.scanFile(filePath); if (isInfected) { throw new Error('Virus detected'); } console.log('AV scan: clean'); // 3. Stream to S3 const s3Key = await uploadToS3(filePath, req.file.originalname, detectedMime); console.log(`Uploaded to S3: ${s3Key}`); // 4. Generate signed download URL const downloadUrl = await generateSignedUrl(s3Key); // Cleanup temp file fs.unlinkSync(filePath); res.json({ success: true, message: 'File uploaded and scanned successfully', downloadUrl, s3Key, mimeType: detectedMime, size: req.file.size, }); } catch (error) { console.error('Upload error:', error.message); if (req.file) fs.unlinkSync(req.file.path); // Cleanup on error res.status(400).json({ error: error.message }); } }); // Health check app.get('/health', (req, res) => res.json({ status: 'ok', clamscan: !!clamscan })); // Start server app.listen(port, () => { console.log(`File Upload Service running on http://localhost:${port}`); console.log(`Upload example: curl -F "file=@test.jpg" "http://localhost:${port}/upload?allowedMimes=image/jpeg,image/png"`); });Docker Setup (Optional, for ClamAV + Service)
# Dockerfile FROM node:18-alpine RUN apk add --no-cache clamav clamav-daemon clamav-libunrar RUN freshclam # Update definitions COPY . /app WORKDIR /app RUN npm install EXPOSE 3000 CMD freshclam && clamav-daemon -d -c /etc/clamav/clamd.conf & node server.jsdocker build -t upload-service . docker run -p 3000:3000 -e UPLOAD_BUCKET=my-upload-bucket -e AWS_ACCESS_KEY_ID=... -e AWS_SECRET_ACCESS_KEY=... upload-serviceUsage Examples
# Upload image (streaming, validated, scanned) curl -F "file=@test.jpg" \ "http://localhost:3000/upload?allowedMimes=image/jpeg,image/png&maxSize=10MB" # Response: # { # "success": true, # "downloadUrl": "https://my-upload-bucket.s3.amazonaws.com/uploads/...?...", # "s3Key": "uploads/...", # "mimeType": "image/jpeg", # "size": 123456 # }Security & Production Notes
- Streaming: Multer streams to disk (no memory buffer for large files).
- MIME: Client Content-Type + server-side magic bytes.
- AV: Real-time via clamd socket. Scale with multiple instances.
- Limits: Enforce
maxSize, bucket policies. - Lifecycle: S3 auto-deletes after 30 days (configurable).
- Signed URLs: Time-limited (1h), HTTPS enforced.
- Scale: For >100MB, use S3 pre-signed multipart uploads + async Lambda scanning (see extensions below).
- Monitoring: Add S3 metrics, CloudWatch Logs.
- Alternatives: VirusTotal API instead of ClamAV (
npm i virustotal-api).
Extensions for Direct S3 Uploads (Advanced)
For client-direct streaming to S3 (no proxy bandwidth):
// Add endpoint /presign-upload app.get('/presign-upload', async (req, res) => { const key = `incoming/${uuidv4()}-${req.query.filename}`; const command = new CreatePresignedPostCommand({ Bucket: bucketName, Key: key, Conditions: [ ['content-length-range', 0, maxFileSize], ['Content-Type', req.query.allowedMimes.split(',')], ], Expires: 3600, }); const url = await createPresignedPost(s3Client, command); res.json(url); });- Set S3 Event Notification -> Lambda for scan/move (use ClamAV Lambda layer: https://github.com/MalwareDev/clamav-layer).
- Poll
/status/:keyfor approval.
This is a production-ready, fully-featured service! 🚀
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.