winget install --id=iamAzeem.spancopy -e
spancopy - a CLI tool to span (copy) files with size threshold
Spancopy is a CLI tool designed to copy files while managing directory sizes based on a specified threshold. It allows users to span (distribute) files across multiple subdirectories when the cumulative size of copied files reaches the defined threshold, ensuring that no single directory exceeds the specified limit.
Key Features:
Audience & Benefit:
Ideal for system administrators, developers, and anyone needing efficient file management. Spancopy helps prevent directory size issues, maintains performance in large-scale storage environments, and organizes files systematically without manual intervention.
Available via winget for Windows users.
spancopy
is a CLI tool to span (copy) files with size threshold from source
to destination
directory. On reaching the threshold
value, the source
files are spanned over multiple subdirectories under the destination
directory.
For more details, pleas refer to the Usage section.
Supported and tested on:
See the CI workflow for more details.
Download the prebuilt binaries from the Releases page.
apt
)# Add repository
echo "deb [trusted=yes] https://iamazeem.github.io/spancopy/apt/$(dpkg --print-architecture)/ ./" | \
sudo tee /etc/apt/sources.list.d/spancopy.list
# Install package
sudo apt update
sudo apt install spancopy
rpm
)# Add repository
sudo tee /etc/yum.repos.d/spancopy.repo << EOF
[spancopy]
name=spancopy
baseurl=https://iamazeem.github.io/spancopy/rpm/\$basearch
enabled=1
gpgcheck=0
EOF
# Install package
sudo yum install spancopy
winget
)# Install with alias
winget.exe install spancopy
# Install with id
winget.exe install --id iamAzeem.spancopy
Run spancopy --help
for usage.
$ spancopy --help
spancopy v0.0.1 - a CLI tool to span (copy) files with size threshold
Usage: spancopy [OPTIONS]
Options:
-h,--help show help and exit
-v,--version show version and exit
-t,--threshold UINT:SIZE [b, kb(=1024b), ...]:POSITIVE REQUIRED
threshold size to span files
-s,--source TEXT:DIR REQUIRED
source directory
-d,--destination TEXT:DIR REQUIRED
destination directory
Notes:
- On all platforms, the `/` is used as the path separator.
- Empty directories are ignored.
- The sizes of all the `source` files must be less than or equal to the
`threshold` value. So, choose a `threshold` value accordingly.
- The `destination` directory must have enough space for all the `source`
files to be copied successfully.
- A main subdirectory under `destination` is created to avoid conflicts.
Following naming convention is used for this subdirectory:
Format: YYYYMMDDTHHMMSSMS
Example: 20220820T170159946
- The main subdirectory under `destination` is removed if it already exists.
- For each `threshold` reached, the file(s) with their complete paths are
copied under their respective subdirectories i.e. 1, 2, 3, and so on.
For example, if for a `source` directory with some files, the `threshold`
is reached three times then there will three subdirectories e.g.:
Main subdir: 20220820T170159946
Threshold 1: 20220820T170159946/1/.../
Threshold 2: 20220820T170159946/2/.../
Threshold 3: 20220820T170159946/3/.../
Examples:
# spancopy from `src` to `dst` with threshold 500 bytes
spancopy --threshold 500b --source ./src --destination ./dst
# spancopy from `src` to `dst` with threshold 100KB
spancopy --threshold 100kb --source ./src --destination ./dst
For any feedback or to report any issues, please open an issue on
GitHub: https://github.com/iamazeem/spancopy/issues
Written by: AZEEM SAJID
Here's a source
directory (file sizes are in bytes for simplicity):
$ tree -hF source/
source/
├── [4.0K] dir1/
│ ├── [ 10] file.10
│ ├── [ 20] file.20
│ ├── [ 30] file.30
│ └── [ 40] file.40
├── [4.0K] dir2/
│ ├── [ 50] file.50
│ ├── [ 60] file.60
│ ├── [ 70] file.70
│ └── [ 80] file.80
├── [4.0K] dir3/
│ └── [ 90] file.90
├── [4.0K] empty-dir/
└── [ 100] file.100
4 directories, 10 files
In the source
directory, the maximum size is 100 bytes so the threshold
value must at least be 100 e.g.:
./spancopy --threshold 100 --source ./source --destination ./destination
Here's the result:
$ tree -hF destination/20220820T170146582/
destination/20220820T170146582/
├── [4.0K] 1/
│ └── [ 100] file.100
├── [4.0K] 2/
│ └── [4.0K] dir1/
│ ├── [ 10] file.10
│ ├── [ 20] file.20
│ ├── [ 30] file.30
│ └── [ 40] file.40
├── [4.0K] 3/
│ └── [4.0K] dir2/
│ └── [ 50] file.50
├── [4.0K] 4/
│ └── [4.0K] dir2/
│ └── [ 60] file.60
├── [4.0K] 5/
│ └── [4.0K] dir2/
│ └── [ 70] file.70
├── [4.0K] 6/
│ └── [4.0K] dir2/
│ └── [ 80] file.80
└── [4.0K] 7/
└── [4.0K] dir3/
└── [ 90] file.90
13 directories, 10 files
Prerequisites:
git clone git@github.com:iamazeem/spancopy.git
cd spancopy
cmake -S . -B build
cmake --build build
The executable will be under the build
directory.
./build/spancopy --help
main
branch.feature
or bugfix
branch from main
.