Untitled conversation
- User
#!/usr/bin/env bash set -euo pipefail PRETOKENIZERS=(bytes_gpt4_cb bytes_gpt4o_cb scriptenc_cb scriptenc_gpt4o_cb) corpora=(CulturaX-subsample-100-bal2) N=256000 N_CPUS=64 for corpus in "${corpora[@]}"; do for pretokenizer in "${PRETOKENIZERS[@]}"; do uv run script_bpe/train.py --report \ -n "$N" \ --corpus "$corpus" \ --pretokenizer "$pretokenizer" \ --parallel "$N_CPUS" | tee log_multilingual.txt done done echo "Done training on: ${corpora[*]} with pretokenizers: ${PRETOKENIZERS[*]}"adapt script to time on 128,64,32,16,8 cpus
Assistant
Okay, here's the adapted bash script that will run your training command for each specified number of CPUs and use the
timecommand to measure how long each run takes.The output will be appended to a log file (
log_timing.txtby default), with clear separators and timestamps indicating the start and end of each specific training run (for a given CPU count, corpus, and pretokenizer). The output from thetimecommand will also be captured in the log file for each run.#!/usr/bin/env bash set -euo pipefail # --- Configuration --- PRETOKENIZERS=(bytes_gpt4_cb bytes_gpt4o_cb scriptenc_cb scriptenc_gpt4o_cb) corpora=(CulturaX-subsample-100-bal2) N=256000 CPU_COUNTS=(128 64 32 16 8) # CPU counts to test LOG_FILE="log_timing.txt" # File to append all output to # --- Script --- echo "--- Starting timing runs ---" | tee -a "$LOG_FILE" echo "Corpora: ${corpora[*]}" | tee -a "$LOG_FILE" echo "Pretokenizers: ${PRETOKENIZERS[*]}" | tee -a "$LOG_FILE" echo "Sample Size (-n): $N" | tee -a "$LOG_FILE" echo "CPU counts to test: ${CPU_COUNTS[*]}" | tee -a "$LOG_FILE" echo "Log file: $LOG_FILE" | tee -a "$LOG_FILE" echo "------------------------------" | tee -a "$LOG_FILE" echo "" | tee -a "$LOG_FILE" # Loop through each CPU count for cpu_count in "${CPU_COUNTS[@]}"; do echo "--- Testing with N_CPUS = ${cpu_count} ---" | tee -a "$LOG_FILE" # Loop through each corpus for corpus in "${corpora[@]}"; do # Loop through each pretokenizer for pretokenizer in "${PRETOKENIZERS[@]}"; do # --- Run Configuration Header --- echo "" | tee -a "$LOG_FILE" echo "--- Running corpus='${corpus}', pretokenizer='${pretokenizer}', parallel='${cpu_count}' ---" | tee -a "$LOG_FILE" echo "Start time: $(date +%Y-%m-%d_%H:%M:%S)" | tee -a "$LOG_FILE" echo "" | tee -a "$LOG_FILE" # ---------------------------------- # Construct the command COMMAND_TO_RUN="uv run script_bpe/train.py --report -n $N --corpus $corpus --pretokenizer $pretokenizer --parallel $cpu_count" # Execute the command and time it. # `time` output goes to stderr, which is then redirected to stdout (2>&1) # and piped to tee, which sends it to both stdout and the log file. # The output of the command itself also goes through the pipe. { time $COMMAND_TO_RUN; } 2>&1 | tee -a "$LOG_FILE" # --- Run Configuration Footer --- echo "" | tee -a "$LOG_FILE" echo "End time: $(date +%Y-%m-%d_%H:%M:%S)" | tee -a "$LOG_FILE" echo "--- Finished corpus='${corpus}', pretokenizer='${pretokenizer}', parallel='${cpu_count}' ---" | tee -a "$LOG_FILE" echo "" | tee -a "$LOG_FILE" # ---------------------------------- done # End pretokenizer loop done # End corpus loop echo "--- Finished testing with N_CPUS = ${cpu_count} ---" | tee -a "$LOG_FILE" echo "" | tee -a "$LOG_FILE" done # End cpu_count loop echo "--- All timing runs completed ---" | tee -a "$LOG_FILE"Explanation:
CPU_COUNTS=(128 64 32 16 8): This new array holds the different values you want to use for the--parallelargument.LOG_FILE="log_timing.txt": Defines the name of the file where all output will be collected.tee -aensures output is appended, not overwritten.- Outer Loop: A new
for cpu_count in "${CPU_COUNTS[@]}"; do ... doneloop is added around your existing loops. This means the entire set of corpus/pretokenizer combinations will be run for each CPU count. time { COMMAND; } 2>&1 | tee -a "$LOG_FILE":time: This built-in bash command (or sometimes an external utility/usr/bin/time) measures the execution time of the following command.{ COMMAND; }: Groups the commanduv run .... This is sometimes useful withtime.2>&1: Redirects the standard error of thetimecommand (where it prints its timing information) and theuv runcommand to standard output.| tee -a "$LOG_FILE": Pipes all standard output (which now includes stderr) to theteecommand.tee -awrites it to both standard output (so you see it in the terminal as it runs) and appends it to the specified log file (log_timing.txt).
- Header/Footer Lines:
echo "--- Running ... ---"andecho "Start/End time: ..."lines are added before and after each individual run within the loops. This makes it much easier to locate and analyze the results for a specific configuration within the combined log file. The| tee -a "$LOG_FILE"ensures these markers also appear in the log file. - Variable Usage: The
--parallelargument now uses the loop variable"$cpu_count".
After running this script, you can examine
log_timing.txt. For each run, you'll see:- The header line indicating the parameters (corpus, pretokenizer, parallel count).
- The start timestamp.
- The standard output/error of the
uv runcommand. - The output from the
timecommand (usually showingreal,user, andsystime). - The end timestamp.
- The footer line.
The
realtime is typically the wall-clock time, which is what you're likely most interested in for performance comparisons.
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.