Hi all, As part of the effort to rewrite the Synapse mediation engine in Go, I've been exploring options for handling file-related operations within the file inbound endpoint. In the current Java implementation of Apache Synapse, the Apache Commons VFS <https://commons.apache.org/proper/commons-vfs/> library is utilized for this purpose. To maintain a similar level of abstraction and flexibility in our Go implementation, we've investigated potential virtual file system (VFS) packages available in the Go ecosystem.
The chosen VFS package should ideally support the following: 1. Common file operations: This includes essential functionalities such as creating, reading, writing, copying, moving, deleting, and listing files and directories. It should also provide mechanisms for checking file existence, retrieving file metadata (size, modification time, permissions), and potentially handling file permissions. 2. Support for FTP and SFTP: Robust support for both FTP (File Transfer Protocol) and SFTP (SSH File Transfer Protocol) is crucial for interacting with various remote file systems. This includes the ability to connect to FTP/SFTP servers, navigate directories, upload and download files, and manage remote file structures. 3. Compatibility across common operating systems: The chosen library should function seamlessly across different operating systems, including Linux, Windows, and macOS, without requiring platform-specific code or configurations. This ensures the portability of the Synapse mediation engine. Beyond these core requirements, we should also consider factors such as the package's maturity, community support, active maintenance, performance characteristics, and ease of integration into our existing codebase. After evaluating several Go VFS packages, including prominent options like spf13/afero <https://github.com/spf13/afero> and go-billy/v3 <https://pkg.go.dev/gopkg.in/src-d/go-billy.v3>, my analysis suggests that c2fo/vfs <https://github.com/C2FO/vfs> presents a compelling choice for our needs. While other packages offer valuable features, c2fo/vfs <https://github.com/C2FO/vfs> stands out due to its well-designed abstraction layer that cleanly separates the file system interface from specific implementations. This design promotes flexibility and testability. It provides support for various backends, including OS files, memory, and importantly, FTP and SFTP. Regarding performance, while comprehensive benchmark data directly comparing all Go VFS packages in all scenarios is limited,the design of c2fo/vfs <https://github.com/C2FO/vfs> suggests it offers a good balance of performance and abstraction. Its focus on providing a consistent API across different file systems minimizes potential performance overhead associated with complex abstractions. For instance, basic file operations like read and write are generally efficient due to the underlying implementations for each supported backend. The c2fo/vfs <https://github.com/C2FO/vfs> package provides a comprehensive set of operations as outlined in its FileSystem <https://pkg.go.dev/github.com/c2fo/vfs/v7#FileSystem>, Location <https://pkg.go.dev/github.com/c2fo/vfs/v7#Location> & File <https://pkg.go.dev/github.com/c2fo/vfs/v7#File> interface. Its support for FTP <https://github.com/C2FO/vfs/blob/main/docs/ftp.md> and SFTP <https://github.com/C2FO/vfs/blob/main/docs/sftp.md> is implemented through separate drivers, allowing for straightforward integration with remote file systems. Its design inherently aims for cross-platform compatibility by abstracting away operating system-specific file handling. Therefore, based on its comprehensive feature set, including support for common file operations, FTP and SFTP, cross-platform compatibility, and a clean architectural design, I propose adopting the c2fo/vfs <https://github.com/C2FO/vfs> package for our future implementation of the file inbound endpoint in the Go-based Synapse mediation engine. I am open to any suggestions or alternative perspectives you might have on this proposal. Please share your thoughts and feedback. -- Thisara Rathnayaka Undergraduate Student | Computer Science & Engineering University of Moratuwa LinkedIn <http://www.linkedin.com/in/weerakoon-thisara-rathnayaka-881826177> | GitHub <https://github.com/ThisaraWeerakoon> | Blog <https://medium.com/@thisara.weerakoon2001>
