Before this change overwriting an existing file with a 0 length file
didn't update the file size.
This change corrects the issue and makes sure the file is truncated
properly.
This was discovered by the full integration tests.
Before this change serve s3 would return NoSuchKey errors when a non
existent prefix was listed.
This change fixes it to return an empty list like AWS does.
This was discovered by the full integration tests.
- Changes
- Rename `--s3-authkey` to `--auth-key` to get it out of the s3 backend namespace
- Enable `Content-MD5` integrity checks
- Remove locking after code audit
- Documentation
- Factor out documentation into seperate file
- Add Quickstart to docs
- Add Bugs section to docs
- Add experimental tag to docs
- Add rclone provider to s3 backend docs
- Fixes
- Correct quirks in s3 backend
- Change fmt.Printlns into fs.Logs
- Make metadata storage per backend not global
- Log on startup if anonymous access is enabled
- Coding style fixes
- rename fs to vfs to save confusion with the rest of rclone code
- rename db to b for *s3Backend
Fixes#7062
- add context to log and fallthrough to error log level
- test: use rclone random lib to generate random strings
- calculate hash from vfs cache if file is uploading
- add server started log with server url
- remove md5 hasher
The upstream library rclone uses for rclone mount no longer supports
freebsd. Not only is it broken, but it no longer compiles.
This patch disables rclone mount for freebsd.
However all is not lost for freebsd users - compiling rclone with the
`cmount` tag, so `go install -tags cmount` will install a working
`rclone mount` command which uses cgofuse and the libfuse C library
directly.
Note that the binaries from rclone.org will not have mount support as
we don't have a freebsd build machine in CI and it is very hard to
cross compile cmount.
See: https://github.com/bazil/fuse/issues/280Fixes#5843
ncdu stores the position that it was in for each directory. However
doing a rescan can cause those positions to be out of range if the
number of files decreased in a directory. When re-entering the
directory, this causes an index out of range error.
This fixes the problem by detecting the index out of range and
flushing the saved directory position.
See: https://forum.rclone.org/t/slice-bounds-out-of-range-during-ncdu/42492/
This was caused by a change to the upstream library
ProtonMail/go-crypto checking the flags on the keys more strictly.
However the signing key for rclone is very old and does not have those
flags. Adding those flags using `gpg --edit-key` and then the
`change-usage` subcommand to remove, save, quite then re-add, save
quit the signing capabilities caused the key to work.
This also adds tests for the verification and adds the selfupdate
tests into the integration test harness as they had been disabled on
CI because they rely on external sources and are sometimes unreliable.
Fixes#7373
With automount the target mount drive appears twice in /proc/self/mountinfo.
379 27 0:70 / /mnt/rclone rw,relatime shared:433 - autofs systemd-1 rw,fd=57,...
566 379 0:90 / /mnt/rclone rw,nosuid,nodev,relatime shared:488 - fuse.rclone remote: rw,...
Before this fix we only looked for the mount once in
/proc/self/mountinfo. It finds the automount line and since this
doesn't have fs type rclone it concludes the mount isn't ready yet.
This patch makes rclone look through all the mounts and if any of them
have fs type rclone it concludes the mount is ready.
See: https://forum.rclone.org/t/systemd-mount-works-but-automount-does-not/42287/
Before this change, if a hardlink command was issued, rclone would
just ignore it and not return an error.
This changes any unknown operations (including hardlink) to return an
unsupported error.
Summary:
In cases where cmount is not available in macOS, we alias nfsmount to mount command and transparently start the NFS server and mount it to the target dir.
The NFS server is started on localhost on a random port so it is reasonably secure.
Test Plan:
```
go run rclone.go mount --http-url https://beta.rclone.org :http: nfs-test
```
Added mount tests:
```
go test ./cmd/nfsmount
```
Summary:
Adding a new command to serve any remote over NFS. This is only useful for new macOS versions where FUSE mounts are not available.
* Added willscot/go-nfs dependency and updated go.mod and go.sum
Test Plan:
```
go run rclone.go serve nfs --http-url https://beta.rclone.org :http:
```
Test that it is serving correctly by mounting the NFS directory.
```
mkdir nfs-test
mount -oport=58654,mountport=58654 localhost: nfs-test
```
Then we can list the mounted directory to see it is working.
```
ls nfs-test
```
This was always the intention, it was just implemented wrong.
This shortens the s3 docs by 1369 bringing them down to half the size
just about.
Fixes#7325
Before this change, bisync ignored the dryRun parameter (only when specified
via the rc.)
This change fixes the issue, so that the dryRun rc parameter is equivalent to
the --dry-run flag.
If the server returns the MIME type as application/octet-stream we
assume it doesn't really know what the MIME type. This patch tries
matching the MIME type from the file extension instead in this case.
This enables the use of servers (like OneDrive for Business) which
don't allow the setting of MIME types on upload and have a poor
selection of mime types.
Fixes#7259
In this commit we introduced a race condition when using the auth
proxy.
94a320f23c serve ftp: update to goftp.io/server v2.0.1
This was due to the re-organisation of the upstream library which made
the driver be a singleton rather than per session.
This means that when using the auth proxy we need to keep track of
which VFS to use by based on which FTP user is connected.
This also adjusts the locking so that the methods will run
concurrently.
Before this change uploading files with rclone to:
rclone serve sftp --vfs-cache-mode full
Would return the error:
command "md5sum XXX" failed with error: unexpected non file
This patch detects that the file is still in the VFS cache and reads
the MD5SUM from there rather from the remote.
Fixes#7241
This adds an additional parameter to the creation of each flag. This
specifies one or more flag groups. This **must** be set for global
flags and **must not** be set for local flags.
This causes flags.md to be built with sections to aid comprehension
and it causes the documentation pages for each command (and the
`--help`) to be built showing the flags groups as specified in the
`groups` annotation on the command.
See: https://forum.rclone.org/t/make-docs-for-mortals-not-only-rclone-gurus/39476/
