Skip to content
The Chat Archive
o1-preview-2024-09-12· OpenAI WildChat-4.8M (AI2)

create a ncdu like program that is entirely python based and aims to modernize the…

  1. User

    create a ncdu like program that is entirely python based and aims to modernize the capabilities of ncdu while providing a cleaner and more aesthetic tub experience

  2. Assistant

    Certainly! Building an ncdu-like program in Python is a fantastic idea. You can leverage Python's capabilities along with modern libraries to create a powerful, user-friendly disk usage analyzer with a sleek terminal interface. Below, I'll guide you through creating such a program, step by step.


    Project Overview

    Goal: Create a Python-based disk usage analyzer similar to ncdu, with a modern and aesthetic text-based user interface (TUI).

    Features to Include:

    • Recursive directory scanning to calculate disk usage.
    • Interactive navigation through directories.
    • Sorting options (e.g., by size or name).
    • Ability to delete files or directories.
    • Option to display or hide hidden files.
    • Clean and modern TUI with enhanced aesthetics.

    Step 1: Set Up the Project Environment

    First, ensure you have Python 3.8 or higher installed. Create a virtual environment for your project:

    python3 -m venv ncdu_python
    source ncdu_python/bin/activate
    

    Install Required Libraries

    We'll use Rich and Textual libraries for the modern TUI:

    pip install rich textual
    
    • Rich: Provides rich text formatting, progress bars, tables, and more for the terminal.
    • Textual: Allows building sophisticated TUIs with widgets and reactive programming patterns.

    Step 2: Plan the Application Structure

    We'll create a class-based application using Textual, which uses an event-driven architecture.

    Application Components:

    • Directory Scanner: Recursively scans directories and calculates sizes.
    • UI Components: Renders the directory tree and handles user interactions.
    • Event Handlers: Respond to user input (e.g., key presses) for navigation and actions.

    Step 3: Implement the Directory Scanner

    We'll start by creating a function to scan directories and compute their sizes.

    import os
    
    def get_directory_size(path):
        total = 0
        try:
            with os.scandir(path) as it:
                for entry in it:
                    if entry.is_symlink():
                        continue  # Skip symbolic links to avoid recursion
                    if entry.is_file():
                        total += entry.stat().st_size
                    elif entry.is_dir():
                        total += get_directory_size(entry.path)
        except PermissionError:
            pass  # Skip directories without permission
        return total
    

    Step 4: Build the TUI with Textual

    We'll create a DiskUsageApp class that extends Textual's App class.

    from textual.app import App, ComposeResult
    from textual.widgets import Tree, TreeNode
    from textual.containers import Container
    from rich.text import Text
    
    class DiskUsageApp(App):
        async def on_mount(self) -> None:
            self.tree = Tree("Loading...", id="disk-tree")
            await self.view.dock(self.tree)
    
            path_to_scan = "/your/path/here"  # Replace with the desired path
            await self.scan_and_build_tree(path_to_scan, self.tree.root)
            self.tree.label = f"Disk Usage for: {path_to_scan}"
            await self.tree.root.expand()
    
        async def scan_and_build_tree(self, path, node: TreeNode):
            try:
                with os.scandir(path) as it:
                    for entry in it:
                        if entry.is_symlink():
                            continue
                        label = entry.name
                        if entry.is_dir():
                            branch = node.add(label, expand=False)
                            await self.scan_and_build_tree(entry.path, branch)
                        else:
                            size = entry.stat().st_size
                            branch_label = f"{label} ({size} bytes)"
                            node.add_leaf(branch_label)
            except PermissionError:
                node.add_leaf(Text("Permission Denied", style="bold red"))
    

    Step 5: Add Interactivity

    Navigating the Tree

    Textual handles event loops and user interactions. The tree widget allows users to navigate using arrow keys. You can enhance this by adding key bindings for additional functionality.

    Implement Sorting

    You can add sorting methods to sort directories/files by size or name.

    def sort_entries(entries, sort_by='size'):
        if sort_by == 'size':
            return sorted(entries, key=lambda e: e.stat().st_size if e.is_file() else get_directory_size(e.path), reverse=True)
        elif sort_by == 'name':
            return sorted(entries, key=lambda e: e.name)
        return entries
    

    Modify the scan_and_build_tree method to incorporate sorting:

    with os.scandir(path) as it:
        entries = list(it)
        entries = sort_entries(entries, sort_by='size')
        for entry in entries:
            # rest of the code
    

    Implement Deletion Functionality

    Add a method to delete selected files or directories.

    from textual import events
    
    class DiskUsageApp(App):
        # existing code...
    
        async def on_key(self, event: events.Key) -> None:
            if event.key == "d":
                await self.delete_selected_node()
    
        async def delete_selected_node(self):
            node = self.tree.focused_node
            if node:
                path = self.get_node_path(node)
                if os.path.isfile(path) or os.path.isdir(path):
                    os.remove(path) if os.path.isfile(path) else os.rmdir(path)
                    node.remove()
                    self.refresh()
        
        def get_node_path(self, node: TreeNode):
            # Reconstruct the path from the node hierarchy
            parts = []
            while node and node.parent:
                parts.insert(0, node.label.plain)
                node = node.parent
            return os.path.join(*parts)
    

    Warning: Be careful with deletion functionality to prevent accidental data loss. Prompt confirmation before deleting.


    Step 6: Enhance Aesthetics with Rich Styling

    You can use Rich's styling features to improve the visual appeal.

    def format_label(entry, size=None):
        if entry.is_dir():
            label = Text(f"{entry.name}/", style="bold blue")
        else:
            label = Text(entry.name)
        if size is not None:
            size_text = Text(f" ({size} bytes)", style="green")
            label.append(size_text)
        return label
    

    Update label assignment in scan_and_build_tree:

    label = format_label(entry)
    if entry.is_dir():
        branch = node.add(label, expand=False)
        await self.scan_and_build_tree(entry.path, branch)
    else:
        size = entry.stat().st_size
        branch_label = format_label(entry, size)
        node.add_leaf(branch_label)
    

    Step 7: Implement Progress Indicators

    For large directories, scanning can take time. Use a progress bar to indicate progress.

    from rich.progress import Progress
    
    async def scan_and_build_tree(self, path, node: TreeNode):
        with Progress() as progress:
            task = progress.add_task(f"Scanning {path}", total=None)
            await self._scan(path, node, progress, task)
    
    async def _scan(self, path, node: TreeNode, progress, task):
        # scanning code, updating progress if possible
    

    However, integrating Rich's Progress with Textual requires careful handling. Refer to the Textual documentation for guidance on integrating progress bars.


    Step 8: Handle Command-line Arguments

    Use argparse to accept command-line arguments for options like the path to scan, sort order, etc.

    import argparse
    
    def parse_arguments():
        parser = argparse.ArgumentParser(description="Python ncdu-like Disk Usage Analyzer")
        parser.add_argument("path", nargs="?", default=".", help="Path to scan")
        parser.add_argument("-s", "--sort", choices=["size", "name"], default="size", help="Sort order")
        return parser.parse_args()
    
    if __name__ == "__main__":
        args = parse_arguments()
        DiskUsageApp.run(title="Disk Usage Analyzer", path=args.path, sort_by=args.sort)
    

    Modify the on_mount method to use self.path and self.sort_by.


    Step 9: Optimize Performance

    For better performance, especially with large filesystems:

    • Use Asynchronous IO: Incorporate asyncio to perform scans asynchronously.
    • Limit Recursion Depth: Provide options to limit the depth of directory scanning.
    • Cache Results: Cache directory sizes to avoid redundant calculations.

    Step 10: Testing and Validation

    Thoroughly test your application:

    • Permissions: Ensure it gracefully handles directories/files without permissions.
    • Large Directories: Test with directories containing many files/subdirectories.
    • Special Files: Handle symbolic links, sockets, etc.

    Finalizing the Application

    Combine all the components, and ensure your application runs smoothly. Here's an outline of your main application file:

    # main.py
    
    import argparse
    from textual.app import App
    # ... (other imports)
    
    class DiskUsageApp(App):
        # ... (implementation as above)
    
    def parse_arguments():
        # ... (as above)
    
    if __name__ == "__main__":
        args = parse_arguments()
        DiskUsageApp.run(title="Disk Usage Analyzer", path=args.path, sort_by=args.sort)
    

    Additional Features to Consider

    • Search Functionality: Implement a search feature to find files/directories.
    • Export Reports: Allow exporting of disk usage reports to a file.
    • Customizable Themes: Enable users to select or customize the color theme.

    Conclusion

    You've now created a Python-based disk usage analyzer with a modern and aesthetic TUI. This application not only replicates ncdu's functionality but also enhances it with modern Python libraries.


    Resources


    Feel free to ask if you need further assistance with specific implementation details or additional features!