File and Folder Copy Management on macOS

Efficiently manage file and folder copying operations across your Roundfleet deployment with enterprise-grade safety features, integrity verification, and comprehensive audit capabilities. This tutorial transforms basic cp commands into robust data distribution solutions.

Understanding Enterprise File Copy Operations

Enterprise file copying goes beyond basic duplication, requiring:

Safety validation to prevent accidental overwrites
Integrity verification to ensure data consistency
Permission preservation for security compliance
Progress monitoring for large operations
Rollback capabilities for failed transfers
Audit logging for compliance requirements

Core Copy Operations

Basic File Copy

#!/bin/bash

# Simple file copy with validation
copy_file() {
    local source="$1"
    local destination="$2"
    
    # Validate source exists
    if [[ ! -f "$source" ]]; then
        echo "Error: Source file '$source' not found"
        return 1
    fi
    
    # Create destination directory if needed
    local dest_dir=$(dirname "$destination")
    mkdir -p "$dest_dir"
    
    # Copy file
    if cp "$source" "$destination"; then
        echo "Successfully copied '$source' to '$destination'"
        return 0
    else
        echo "Failed to copy '$source' to '$destination'"
        return 1
    fi
}

# Usage example
# copy_file "/Users/admin/document.pdf" "/Users/shared/documents/document.pdf"

Basic Directory Copy

#!/bin/bash

# Recursive directory copy with verification
copy_directory() {
    local source="$1"
    local destination="$2"
    
    # Validate source directory exists
    if [[ ! -d "$source" ]]; then
        echo "Error: Source directory '$source' not found"
        return 1
    fi
    
    # Copy directory recursively
    if cp -R "$source" "$destination"; then
        echo "Successfully copied directory '$source' to '$destination'"
        return 0
    else
        echo "Failed to copy directory '$source' to '$destination'"
        return 1
    fi
}

# Usage example
# copy_directory "/Users/admin/project" "/Users/shared/projects/"

Enterprise Copy Management System

#!/bin/bash

# Roundfleet Enterprise File Copy Management System
# Comprehensive file and folder copying with enterprise features

# Configuration
SCRIPT_NAME="Roundfleet Copy Manager"
VERSION="1.0.0"
LOG_FILE="/var/log/roundfleet_copy_operations.log"
TEMP_DIR="/tmp/roundfleet_copy"
MAX_FILE_SIZE="10G"
ALLOWED_EXTENSIONS=(".pdf" ".docx" ".xlsx" ".pptx" ".txt" ".png" ".jpg" ".gif" ".mp4" ".mov")
RESTRICTED_PATHS=("/System" "/usr/bin" "/usr/sbin" "/private/var")
BUSINESS_HOURS_START=9
BUSINESS_HOURS_END=17

# Create necessary directories
mkdir -p "$TEMP_DIR"
mkdir -p "$(dirname "$LOG_FILE")"

# Logging function
log_operation() {
    local level="$1"
    local message="$2"
    local timestamp=$(date '+%Y-%m-%d %H:%M:%S')
    echo "[$timestamp] [$level] $message" | tee -a "$LOG_FILE"
}

# Check if current time is within business hours
is_business_hours() {
    local current_hour=$(date +%H)
    if [[ $current_hour -ge $BUSINESS_HOURS_START && $current_hour -lt $BUSINESS_HOURS_END ]]; then
        return 0
    else
        return 1
    fi
}

# Validate file extension
is_allowed_extension() {
    local file="$1"
    local extension="${file##*.}"
    extension=".$extension"
    
    for allowed in "${ALLOWED_EXTENSIONS[@]}"; do
        if [[ "$extension" == "$allowed" ]]; then
            return 0
        fi
    done
    return 1
}

# Check if path is restricted
is_restricted_path() {
    local path="$1"
    for restricted in "${RESTRICTED_PATHS[@]}"; do
        if [[ "$path" == "$restricted"* ]]; then
            return 0
        fi
    done
    return 1
}

# Get file size in human readable format
get_file_size() {
    local file="$1"
    if [[ -f "$file" ]]; then
        stat -f%z "$file" 2>/dev/null
    elif [[ -d "$file" ]]; then
        du -sk "$file" 2>/dev/null | awk '{print $1 * 1024}'
    else
        echo "0"
    fi
}

# Convert bytes to human readable format
format_size() {
    local bytes="$1"
    local sizes=("B" "KB" "MB" "GB" "TB")
    local unit=0
    
    while [[ $bytes -gt 1024 && $unit -lt 4 ]]; do
        bytes=$((bytes / 1024))
        ((unit++))
    done
    
    echo "${bytes}${sizes[$unit]}"
}

# Calculate checksum for integrity verification
calculate_checksum() {
    local file="$1"
    if [[ -f "$file" ]]; then
        shasum -a 256 "$file" 2>/dev/null | awk '{print $1}'
    else
        echo ""
    fi
}

# Verify copy integrity
verify_copy_integrity() {
    local source="$1"
    local destination="$2"
    
    if [[ -f "$source" && -f "$destination" ]]; then
        local source_checksum=$(calculate_checksum "$source")
        local dest_checksum=$(calculate_checksum "$destination")
        
        if [[ "$source_checksum" == "$dest_checksum" ]]; then
            log_operation "INFO" "Integrity verification passed for: $destination"
            return 0
        else
            log_operation "ERROR" "Integrity verification failed for: $destination"
            return 1
        fi
    elif [[ -d "$source" && -d "$destination" ]]; then
        # For directories, compare file counts and sizes
        local source_count=$(find "$source" -type f | wc -l)
        local dest_count=$(find "$destination" -type f | wc -l)
        
        if [[ $source_count -eq $dest_count ]]; then
            log_operation "INFO" "Directory integrity verification passed for: $destination"
            return 0
        else
            log_operation "ERROR" "Directory integrity verification failed for: $destination"
            return 1
        fi
    else
        log_operation "ERROR" "Cannot verify integrity - source or destination missing"
        return 1
    fi
}

# Create backup before overwrite
create_backup() {
    local file="$1"
    local backup_dir="/var/backups/roundfleet/copy_operations"
    local timestamp=$(date '+%Y%m%d_%H%M%S')
    
    mkdir -p "$backup_dir"
    
    if [[ -e "$file" ]]; then
        local filename=$(basename "$file")
        local backup_file="$backup_dir/${filename}_backup_$timestamp"
        
        if cp -R "$file" "$backup_file"; then
            log_operation "INFO" "Backup created: $backup_file"
            echo "$backup_file"
            return 0
        else
            log_operation "ERROR" "Failed to create backup for: $file"
            return 1
        fi
    fi
}

# Enhanced file copy with enterprise features
enterprise_copy_file() {
    local source="$1"
    local destination="$2"
    local preserve_permissions="${3:-true}"
    local verify_integrity="${4:-true}"
    local create_backup_flag="${5:-true}"
    
    log_operation "INFO" "Starting file copy operation: $source -> $destination"
    
    # Pre-flight validations
    if [[ ! -f "$source" ]]; then
        log_operation "ERROR" "Source file not found: $source"
        return 1
    fi
    
    # Check if source is restricted
    if is_restricted_path "$source"; then
        log_operation "ERROR" "Source path is restricted: $source"
        return 1
    fi
    
    # Check if destination is restricted
    if is_restricted_path "$destination"; then
        log_operation "ERROR" "Destination path is restricted: $destination"
        return 1
    fi
    
    # Validate file extension
    if ! is_allowed_extension "$source"; then
        log_operation "WARNING" "File extension not in allowed list: $source"
    fi
    
    # Check file size
    local file_size=$(get_file_size "$source")
    local max_size_bytes=$(echo "$MAX_FILE_SIZE" | sed 's/G//' | awk '{print $1 * 1024 * 1024 * 1024}')
    
    if [[ $file_size -gt $max_size_bytes ]]; then
        log_operation "ERROR" "File too large: $(format_size $file_size) > $MAX_FILE_SIZE"
        return 1
    fi
    
    # Business hours check for large files (>100MB)
    if [[ $file_size -gt 104857600 ]] && ! is_business_hours; then
        log_operation "WARNING" "Large file copy outside business hours: $(format_size $file_size)"
    fi
    
    # Create destination directory
    local dest_dir=$(dirname "$destination")
    if ! mkdir -p "$dest_dir"; then
        log_operation "ERROR" "Failed to create destination directory: $dest_dir"
        return 1
    fi
    
    # Create backup if destination exists
    local backup_file=""
    if [[ -f "$destination" && "$create_backup_flag" == "true" ]]; then
        backup_file=$(create_backup "$destination")
        if [[ $? -ne 0 ]]; then
            log_operation "ERROR" "Failed to create backup for existing file: $destination"
            return 1
        fi
    fi
    
    # Perform the copy operation
    local copy_options=""
    if [[ "$preserve_permissions" == "true" ]]; then
        copy_options="-p"
    fi
    
    if cp $copy_options "$source" "$destination"; then
        log_operation "INFO" "File copied successfully: $(format_size $file_size)"
        
        # Verify integrity if requested
        if [[ "$verify_integrity" == "true" ]]; then
            if ! verify_copy_integrity "$source" "$destination"; then
                log_operation "ERROR" "Copy integrity verification failed, removing destination"
                rm -f "$destination"
                
                # Restore backup if it exists
                if [[ -n "$backup_file" && -f "$backup_file" ]]; then
                    cp "$backup_file" "$destination"
                    log_operation "INFO" "Restored backup file: $destination"
                fi
                return 1
            fi
        fi
        
        # Log successful operation
        log_operation "SUCCESS" "File copy completed: $source -> $destination"
        return 0
    else
        log_operation "ERROR" "Copy operation failed: $source -> $destination"
        
        # Restore backup if it exists
        if [[ -n "$backup_file" && -f "$backup_file" ]]; then
            cp "$backup_file" "$destination"
            log_operation "INFO" "Restored backup file: $destination"
        fi
        return 1
    fi
}

# Enhanced directory copy with enterprise features
enterprise_copy_directory() {
    local source="$1"
    local destination="$2"
    local preserve_permissions="${3:-true}"
    local verify_integrity="${4:-true}"
    local create_backup_flag="${5:-true}"
    
    log_operation "INFO" "Starting directory copy operation: $source -> $destination"
    
    # Pre-flight validations
    if [[ ! -d "$source" ]]; then
        log_operation "ERROR" "Source directory not found: $source"
        return 1
    fi
    
    # Check if source is restricted
    if is_restricted_path "$source"; then
        log_operation "ERROR" "Source path is restricted: $source"
        return 1
    fi
    
    # Check if destination is restricted
    if is_restricted_path "$destination"; then
        log_operation "ERROR" "Destination path is restricted: $destination"
        return 1
    fi
    
    # Calculate directory size
    local dir_size=$(get_file_size "$source")
    local file_count=$(find "$source" -type f | wc -l)
    
    log_operation "INFO" "Directory stats - Size: $(format_size $dir_size), Files: $file_count"
    
    # Business hours check for large directories (>1GB or >1000 files)
    if [[ $dir_size -gt 1073741824 || $file_count -gt 1000 ]] && ! is_business_hours; then
        log_operation "WARNING" "Large directory copy outside business hours"
    fi
    
    # Create backup if destination exists
    local backup_dir=""
    if [[ -d "$destination" && "$create_backup_flag" == "true" ]]; then
        backup_dir=$(create_backup "$destination")
        if [[ $? -ne 0 ]]; then
            log_operation "ERROR" "Failed to create backup for existing directory: $destination"
            return 1
        fi
    fi
    
    # Perform the copy operation
    local copy_options="-R"
    if [[ "$preserve_permissions" == "true" ]]; then
        copy_options="-Rp"
    fi
    
    if cp $copy_options "$source" "$destination"; then
        log_operation "INFO" "Directory copied successfully: $(format_size $dir_size)"
        
        # Verify integrity if requested
        if [[ "$verify_integrity" == "true" ]]; then
            if ! verify_copy_integrity "$source" "$destination"; then
                log_operation "ERROR" "Directory copy integrity verification failed, removing destination"
                rm -rf "$destination"
                
                # Restore backup if it exists
                if [[ -n "$backup_dir" && -d "$backup_dir" ]]; then
                    cp -R "$backup_dir" "$destination"
                    log_operation "INFO" "Restored backup directory: $destination"
                fi
                return 1
            fi
        fi
        
        # Log successful operation
        log_operation "SUCCESS" "Directory copy completed: $source -> $destination"
        return 0
    else
        log_operation "ERROR" "Directory copy operation failed: $source -> $destination"
        
        # Restore backup if it exists
        if [[ -n "$backup_dir" && -d "$backup_dir" ]]; then
            cp -R "$backup_dir" "$destination"
            log_operation "INFO" "Restored backup directory: $destination"
        fi
        return 1
    fi
}

# Bulk copy operations with progress monitoring
bulk_copy_operation() {
    local operation_type="$1"  # "file" or "directory"
    local source_list="$2"     # File containing source paths
    local destination_base="$3" # Base destination directory
    
    if [[ ! -f "$source_list" ]]; then
        log_operation "ERROR" "Source list file not found: $source_list"
        return 1
    fi
    
    local total_items=$(wc -l < "$source_list")
    local current_item=0
    local success_count=0
    local failure_count=0
    
    log_operation "INFO" "Starting bulk copy operation - Total items: $total_items"
    
    while IFS= read -r source_path; do
        ((current_item++))
        
        # Skip empty lines and comments
        [[ -z "$source_path" || "$source_path" =~ ^#.* ]] && continue
        
        local filename=$(basename "$source_path")
        local destination="$destination_base/$filename"
        
        echo "Processing [$current_item/$total_items]: $filename"
        
        if [[ "$operation_type" == "file" ]]; then
            if enterprise_copy_file "$source_path" "$destination"; then
                ((success_count++))
            else
                ((failure_count++))
            fi
        elif [[ "$operation_type" == "directory" ]]; then
            if enterprise_copy_directory "$source_path" "$destination"; then
                ((success_count++))
            else
                ((failure_count++))
            fi
        fi
        
        # Progress update
        local progress=$((current_item * 100 / total_items))
        echo "Progress: $progress% ($success_count successful, $failure_count failed)"
        
    done < "$source_list"
    
    log_operation "SUCCESS" "Bulk copy completed - Success: $success_count, Failed: $failure_count"
    return $failure_count
}

# Generate copy operation report
generate_copy_report() {
    local report_file="/tmp/roundfleet_copy_report_$(date +%Y%m%d_%H%M%S).txt"
    
    {
        echo "Roundfleet Copy Operations Report"
        echo "Generated: $(date)"
        echo "Hostname: $(hostname)"
        echo "=============================="
        echo ""
        
        echo "Recent Copy Operations (Last 24 hours):"
        if [[ -f "$LOG_FILE" ]]; then
            local yesterday=$(date -v-1d '+%Y-%m-%d')
            grep "$yesterday\|$(date '+%Y-%m-%d')" "$LOG_FILE" | tail -50
        else
            echo "No log file found"
        fi
        
        echo ""
        echo "System Information:"
        echo "Available Space:"
        df -h | grep -E "^/dev/"
        
        echo ""
        echo "Copy Operation Statistics:"
        if [[ -f "$LOG_FILE" ]]; then
            echo "Total Operations: $(grep -c "copy operation" "$LOG_FILE" 2>/dev/null || echo "0")"
            echo "Successful: $(grep -c "SUCCESS.*copy completed" "$LOG_FILE" 2>/dev/null || echo "0")"
            echo "Failed: $(grep -c "ERROR.*copy.*failed" "$LOG_FILE" 2>/dev/null || echo "0")"
        fi
        
    } > "$report_file"
    
    echo "Copy operations report saved to: $report_file"
    log_operation "INFO" "Copy report generated: $report_file"
}

# Main copy management function
main() {
    local action="${1:-help}"
    
    case "$action" in
        "copy-file")
            local source="$2"
            local destination="$3"
            local preserve_perms="${4:-true}"
            local verify="${5:-true}"
            local backup="${6:-true}"
            
            if [[ -z "$source" || -z "$destination" ]]; then
                echo "Usage: $0 copy-file <source> <destination> [preserve_permissions] [verify_integrity] [create_backup]"
                exit 1
            fi
            
            enterprise_copy_file "$source" "$destination" "$preserve_perms" "$verify" "$backup"
            ;;
        "copy-directory")
            local source="$2"
            local destination="$3"
            local preserve_perms="${4:-true}"
            local verify="${5:-true}"
            local backup="${6:-true}"
            
            if [[ -z "$source" || -z "$destination" ]]; then
                echo "Usage: $0 copy-directory <source> <destination> [preserve_permissions] [verify_integrity] [create_backup]"
                exit 1
            fi
            
            enterprise_copy_directory "$source" "$destination" "$preserve_perms" "$verify" "$backup"
            ;;
        "bulk-files")
            local source_list="$2"
            local destination="$3"
            
            if [[ -z "$source_list" || -z "$destination" ]]; then
                echo "Usage: $0 bulk-files <source_list_file> <destination_directory>"
                exit 1
            fi
            
            bulk_copy_operation "file" "$source_list" "$destination"
            ;;
        "bulk-directories")
            local source_list="$2"
            local destination="$3"
            
            if [[ -z "$source_list" || -z "$destination" ]]; then
                echo "Usage: $0 bulk-directories <source_list_file> <destination_directory>"
                exit 1
            fi
            
            bulk_copy_operation "directory" "$source_list" "$destination"
            ;;
        "report")
            generate_copy_report
            ;;
        "help"|*)
            echo "$SCRIPT_NAME v$VERSION"
            echo "Enterprise File and Folder Copy Management"
            echo ""
            echo "Usage: $0 <action> [options]"
            echo ""
            echo "Actions:"
            echo "  copy-file <source> <destination>       - Copy single file"
            echo "  copy-directory <source> <destination>  - Copy directory recursively"
            echo "  bulk-files <list_file> <destination>   - Bulk copy files from list"
            echo "  bulk-directories <list_file> <dest>    - Bulk copy directories from list"
            echo "  report                                  - Generate operations report"
            echo "  help                                    - Show this help message"
            echo ""
            echo "Features:"
            echo "  • Safety validation and integrity verification"
            echo "  • Automatic backup creation before overwrite"
            echo "  • Business hours compliance checking"
            echo "  • Comprehensive audit logging"
            echo "  • Permission preservation"
            echo "  • Progress monitoring for bulk operations"
            ;;
    esac
}

# Execute main function with all arguments
main "$@"

Quick Reference Commands

Single File Operations

# Copy file with full enterprise features
./copy_manager.sh copy-file "/Users/admin/document.pdf" "/Users/shared/documents/document.pdf"

# Copy file without backup creation
./copy_manager.sh copy-file "/source/file.txt" "/destination/file.txt" true true false

# Copy file without integrity verification (faster)
./copy_manager.sh copy-file "/source/file.txt" "/destination/file.txt" true false true

Directory Operations

# Copy directory with all subdirectories
./copy_manager.sh copy-directory "/Users/admin/project" "/Users/shared/projects/"

# Copy directory preserving permissions
./copy_manager.sh copy-directory "/source/folder" "/destination/" true true true

Bulk Operations

# Create file list for bulk operations
echo "/Users/admin/doc1.pdf" > /tmp/files_to_copy.txt
echo "/Users/admin/doc2.pdf" >> /tmp/files_to_copy.txt
echo "/Users/admin/doc3.pdf" >> /tmp/files_to_copy.txt

# Execute bulk file copy
./copy_manager.sh bulk-files "/tmp/files_to_copy.txt" "/Users/shared/documents/"

# Execute bulk directory copy
./copy_manager.sh bulk-directories "/tmp/dirs_to_copy.txt" "/Users/shared/projects/"

Integration Examples

JAMF Pro Integration

#!/bin/bash

# JAMF Pro script for enterprise file distribution
# Parameters: $4 = source_path, $5 = destination_path, $6 = operation_type

SOURCE_PATH="$4"
DESTINATION_PATH="$5"
OPERATION_TYPE="$6"

# Download copy manager if not present
if [[ ! -f "/usr/local/bin/roundfleet_copy_manager.sh" ]]; then
    curl -o "/usr/local/bin/roundfleet_copy_manager.sh" "https://scripts.roundfleet.com/copy_manager.sh"
    chmod +x "/usr/local/bin/roundfleet_copy_manager.sh"
fi

# Execute copy operation
case "$OPERATION_TYPE" in
    "file")
        /usr/local/bin/roundfleet_copy_manager.sh copy-file "$SOURCE_PATH" "$DESTINATION_PATH"
        ;;
    "directory")
        /usr/local/bin/roundfleet_copy_manager.sh copy-directory "$SOURCE_PATH" "$DESTINATION_PATH"
        ;;
    *)
        echo "Invalid operation type: $OPERATION_TYPE"
        exit 1
        ;;
esac

# Generate report
/usr/local/bin/roundfleet_copy_manager.sh report

exit $?

Configuration Profile for Copy Policies

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>PayloadContent</key>
    <array>
        <dict>
            <key>PayloadType</key>
            <string>com.roundfleet.copy.policy</string>
            <key>PayloadIdentifier</key>
            <string>com.roundfleet.copy.policy.main</string>
            <key>PayloadDisplayName</key>
            <string>Roundfleet Copy Policy</string>
            <key>MaxFileSize</key>
            <string>10G</string>
            <key>AllowedExtensions</key>
            <array>
                <string>.pdf</string>
                <string>.docx</string>
                <string>.xlsx</string>
                <string>.pptx</string>
            </array>
            <key>BusinessHoursOnly</key>
            <true/>
            <key>RequireIntegrityCheck</key>
            <true/>
            <key>CreateBackups</key>
            <true/>
        </dict>
    </array>
</dict>
</plist>

Security and Compliance Features

File System Monitoring

# Monitor copy operations with FSEvents
fswatch -r /Users/shared/ | while read event; do
    if [[ "$event" =~ "CREATED" ]]; then
        log_operation "MONITOR" "File created: $event"
    fi
done

Compliance Reporting

# Generate compliance report for auditors
generate_compliance_report() {
    local report_file="/var/reports/copy_compliance_$(date +%Y%m%d).csv"
    
    echo "Timestamp,User,Source,Destination,Size,Status,Checksum" > "$report_file"
    
    # Parse log file for copy operations
    grep "copy completed\|copy.*failed" "$LOG_FILE" | while read line; do
        # Extract relevant information and format as CSV
        echo "$line" | awk -F' - ' '{print $1","$2}' >> "$report_file"
    done
    
    echo "Compliance report generated: $report_file"
}

Troubleshooting

Common Issues and Solutions

Issue	Cause	Solution
Permission denied	Insufficient privileges	Run with sudo or check file permissions
Disk space error	Destination full	Check available space with `df -h`
Integrity check failed	Corrupt copy	Re-run copy operation, check disk health
Source not found	Incorrect path	Verify source path exists
Operation timeout	Large file/slow disk	Increase timeout or split operation

Log Analysis

# View recent copy operations
tail -f /var/log/roundfleet_copy_operations.log

# Search for failed operations
grep "ERROR" /var/log/roundfleet_copy_operations.log

# Count operations by status
grep -c "SUCCESS" /var/log/roundfleet_copy_operations.log
grep -c "ERROR" /var/log/roundfleet_copy_operations.log

Best Practices

Always test on single device before fleet deployment
Monitor disk space before large copy operations
Use business hours restrictions for resource-intensive operations
Enable integrity verification for critical data
Maintain regular backups of important destinations
Review logs regularly for failed operations
Set appropriate file size limits based on network capacity
Use bulk operations for efficiency at scale

This enterprise copy management system provides comprehensive safety, monitoring, and compliance features while maintaining the simplicity of basic copy operations for day-to-day fleet management tasks.

Tutorial

Important notice