I've never used NCO before. Can you show me how to make a shell script to run these…
- User
I've never used NCO before. Can you show me how to make a shell script to run these commands from?
Assistant
Here's a robust shell script template for NCO processing:
#!/bin/bash # NCO Yearly File Concatenation Script # Usage: ./concat_yearly.sh # ===== Configuration ===== # Modify these paths to match your specific setup BASE_DIR="/dfs9/baldwij1-lab/hstaudmy/chapter1/output/ssp585/ACCESS-CM2/04_apply_deltas_and_trim_to_90/ssp585/onepointfive" OUTPUT_DIR="${BASE_DIR}/yearly_files" # ===== Logging Setup ===== LOG_FILE="${BASE_DIR}/nco_concat.log" exec > >(tee -a "$LOG_FILE") 2>&1 # ===== Preparation ===== # Create output directory if it doesn't exist mkdir -p "$OUTPUT_DIR" # ===== Main Processing ===== echo "Starting NCO yearly concatenation: $(date)" echo "Source directory: $BASE_DIR" echo "Output directory: $OUTPUT_DIR" # Loop through years for year in $(seq 1980 2010); do echo "Processing year: $year" # Find all files for this year input_files=$(find "$BASE_DIR" -type f -name "*${year}*.nc") # Check if files exist for this year if [ -z "$input_files" ]; then echo "No files found for year $year. Skipping." continue fi # Output filename output_file="${OUTPUT_DIR}/ACCESS-CM2_ssp585_onepointfive_${year}.nc" # Concatenate files ncrcat -O -h $input_files "$output_file" # Check if concatenation was successful if [ $? -eq 0 ]; then echo "Successfully created $output_file" else echo "ERROR: Failed to process files for year $year" fi done # ===== Completion ===== echo "NCO concatenation complete: $(date)" echo "Output files are in $OUTPUT_DIR" # Optional: Verify output files echo "Verifying output files:" ls -lh "$OUTPUT_DIR"🚀 How to Use:
- Save the script:
nano concat_yearly.sh- Make it executable:
chmod +x concat_yearly.sh- Run the script:
./concat_yearly.sh🛠 Script Features:
- Automatic logging to a log file
- Error handling
- Flexible path configuration
- Timestamp tracking
- Output verification
🔍 NCO Command Breakdown:
ncrcat: NetCDF Record Concatenate-O: Overwrite existing output files-h: Speed up by skipping some metadata checks
💡 Pro Tips:
- Modify
BASE_DIRand paths to match your exact file structure - The script uses
findto locate files, so it's more flexible with filename variations - Logs everything to help with troubleshooting
🚨 Potential Modifications:
- Add more error checking
- Implement parallel processing if you have many files
- Add checksum verification
Would you like me to elaborate on any part of the script or explain any specific feature in more detail?
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.