Commit Graph

2200 Commits

Author SHA1 Message Date
Nick Craig-Wood
1afac32d80 serve restic: script for running integration test against all remotes 2018-03-18 19:15:39 +00:00
Nick Craig-Wood
26fbd00b4f serve restic: don't buffer the JSON output in memory for the list command 2018-03-18 16:26:58 +00:00
Nick Craig-Wood
1313b529ff serve restic: use ListR (--fast-list) if available
For Restic's use case, --fast-list will use less transactions and
calling ListR directly means we can avoid the usual memory overhead.
2018-03-18 16:22:05 +00:00
Nick Craig-Wood
82e835d6fc serve restic: make it easy to run integration tests against any remote
Just `cd cmd/serve/restic` then `go test -v -remote TestRemote:`
2018-03-18 14:23:56 +00:00
Nick Craig-Wood
fa867a9a4c serve restic: implement accounting for uploads and downloads
This means the bandwidth stats will be correct and the bandwidth
throttling will work correctly.  This was forgotten as a previous
iteration of the code was using the higher level operations.Rcat which
took care of this.
2018-03-18 14:19:43 +00:00
Nick Craig-Wood
38d9475a34 release: include a source tarball and sign everything #1449 2018-03-17 15:06:04 +00:00
Nick Craig-Wood
c21c7e75b0 Add Stefan Lindblom to contributors 2018-03-17 12:12:23 +00:00
Stefan Lindblom
c8d095612a drive: Document process for service account and impersonation 2018-03-17 12:11:48 +00:00
Nick Craig-Wood
012d4a1235 docs: fix download icon 2018-03-17 12:00:14 +00:00
Nick Craig-Wood
854d3c3025 Add Dave Pedu to contributors 2018-03-17 12:00:14 +00:00
Dave Pedu
5bedc4c668 crypt: fix path in docs 2018-03-17 11:59:25 +00:00
Stefan
86892467d9
config: load config file only on first access (closes #1659, closes #2096) (#2147) 2018-03-17 12:36:30 +01:00
Nick Craig-Wood
e62fe06763 s3: document --ignore-checksum workaround for KMS #1824 2018-03-17 10:51:45 +00:00
Nick Craig-Wood
4295428a0f fs/accounting: add remote control of bwlimit 2018-03-17 10:34:02 +00:00
Nick Craig-Wood
2db0c4dd95 vfs: add remote control for directory cache flushing 2018-03-17 10:34:02 +00:00
Nick Craig-Wood
5bf639048f sync: log an error that --track-renames doesn't work with sync or move
Fixes #2008
2018-03-17 10:34:02 +00:00
remusb
4924ac2f17 cache: reduce log level for plex api - for #2102 2018-03-17 11:57:36 +02:00
Nick Craig-Wood
d4cca8d9f9 onedrive: fix upload of zero length files #1716
Unfortunately multi part upload can't upload zero length files so
bring back the single part upload for zero length files only.

This was broken when we made all uploads multipart uploads.
2018-03-17 09:48:28 +00:00
Nick Craig-Wood
a9e386b153 Add wolfv to contributors 2018-03-17 09:06:51 +00:00
wolfv
117238211b docs: Change log levels to all caps - fixes #2101 2018-03-17 09:06:51 +00:00
Oliver Heyme
645cf5ec0f onedrive: fix wrong upload endpoint and createDate #1716
This fixes the problem introduced by 7f744033d8
2018-03-16 19:18:51 +00:00
Nick Craig-Wood
d1bb8efb88 sftp: follow symlinks correctly - fixes #2145
The sftp library delivers the attributes of the symlink rather than
the object pointed to in directory listings, however when we use Stat
from the library it points to the objects.

Previous to this fix this caused items pointed to by symlinks to be
unusable.

After the fix both symlinked files and directories work as expected.
2018-03-16 15:36:47 +00:00
Nick Craig-Wood
c19e675ca6 vfs: unify locking for RWFileHandle.openPending,.close and File.Delete #2141
Without this fix the cached file can be removed as the file is being
uploaded or downloaded.  This can cause the directory listings to
become inconsistent (this issue) or data loss (if a retry was needed
in the Copy).

Remove file needs to be excluded from running at the same time as both
openPending and close so it makes sense to unify the locking between
all 3.
2018-03-15 20:49:07 +00:00
Nick Craig-Wood
34c45a7c04 mount, cmount: remove addition of O_CREATE to flags on file open #2141
Previously this was adding it in to all file opens which was causing
inefficiencies under Windows where it stats the file using
open/fstat/close.

This change will make stat operations run much quicker under Windows
as they won't have to open the underlying file

This problem was introduced in61b6159a05336bd7ba105766de2d2ff171f7fb81
where we added O_CREATE to all file opens and creates.
2018-03-15 20:48:56 +00:00
Nick Craig-Wood
0a0318df20 Add Leo R. Lundgren to contributors 2018-03-15 20:24:42 +00:00
Leo R. Lundgren
04e055fc06 sftp: Add --sftp-ask-password flag to prompt for password when needed - #2137 2018-03-15 20:24:30 +00:00
Nick Craig-Wood
d551137635 Add Giri Badanahatti to contributors 2018-03-15 20:21:12 +00:00
Giri Badanahatti
aba43cd3a4 Documention for IBM COS (S3) configuration. 2018-03-15 20:20:43 +00:00
Oliver Heyme
7f744033d8 onedrive: Removed upload cutoff and always do session uploads
Set modtime on copy


Added versioning issue to OneDrive documentation
2018-03-15 20:18:11 +00:00
remusb
078d705dbe cache: notify vfs and support crypt in rpc - #2111 2018-03-15 11:39:16 +02:00
Nick Craig-Wood
5981f9fab5 acd: disable integration tests
We no longer have any working keys for Amazon Cloud Drive so disable
the integration tests.
2018-03-14 22:44:46 +00:00
Alexander Neumann
84776c4e43 serve/restic: Remove log message on Close 2018-03-14 21:50:33 +00:00
Nick Craig-Wood
c1a3e363a6 mount: return ENOSYS rather than EIO on attempted link
This fixes FileZilla accessing an rclone mount served over sftp.

See: https://forum.rclone.org/t/moving-files-on-rclone-mount-with-filezilla/5029
2018-03-14 21:10:20 +00:00
Nick Craig-Wood
7ccc6080b0 serve restic: add more info to GET request error 2018-03-14 21:09:47 +00:00
remusb
677971643c cache: add support for rc 2018-03-14 22:58:20 +02:00
remusb
f4a1c1163c rc: update doc with supported params 2018-03-14 22:58:20 +02:00
remusb
97b48cf988 rc: add support for Go 1.6 2018-03-14 22:58:20 +02:00
Nick Craig-Wood
86e5a35491 Implement Remote Control for rclone #2111
This implements a remote control protocol activated with the --rc flag
and a new command `rclone rc` to use that interface.

Still to do
  * docs - need finishing
  * tests
2018-03-14 22:58:20 +02:00
Nick Craig-Wood
8bb2854fe4 httplib: allow the flags to be prefixed when instantiating a server 2018-03-14 22:58:20 +02:00
Remus Bunduc
d76da1f5fd cache: fix dir cache issue - #2117 2018-03-14 11:08:30 +02:00
Nick Craig-Wood
89748feaa5 s3: update docs to discourage use of v2 auth - fixes #2120
From testing it appears that CEPH no longer works properly with v2
auth and neither does Dreamhost, so update the docs anc configuration
to recommend v4 auth.
2018-03-13 20:47:29 +00:00
Nick Craig-Wood
dfd0f4c5a4 sync: when using --backup-dir don't delete files if we can't set their modtime
This is a problem when syncing a file which just needed its modtime
set with dropbox which can't set the mod time of a file without
re-uploading it.

Before this change we would delete the file, then the server side move
would fail moving the file to the backup-dir because it no longer
existed.

After this change the destination file is moved to the backup-dir
instead of being deleted and the new file is uploaded.

Fixes #2134
2018-03-13 16:05:06 +00:00
Nick Craig-Wood
0c9dc006c5 fs: make display of default values of --min-age/--max-age be off - Fixes #2121 2018-03-13 09:06:07 +00:00
Nick Craig-Wood
4e90ad04d5 serve restic: only accept v2 API requests for list 2018-03-11 17:35:01 +00:00
Nick Craig-Wood
43c7ea81df Add Alexander Neumann to contributors 2018-03-11 17:35:01 +00:00
Alexander Neumann
fa003e89b6 serve restic: When listing return empty list instead of 'null' 2018-03-11 14:48:46 +00:00
Alexander Neumann
5114b11d6f serve restic: add http2 server on stdin/stdout 2018-03-11 14:48:46 +00:00
Alexander Neumann
f832433fa5 serve restic: Return empty list for non-existing dirs 2018-03-11 14:48:43 +00:00
Nick Craig-Wood
d073efdc6c serve restic: serves a remote in restic REST API format 2018-03-11 14:43:03 +00:00
Nick Craig-Wood
9e48748182 httplib: Note that authentication is a good idea for non localhost 2018-03-11 14:38:54 +00:00