Commit Graph

839 Commits

Author SHA1 Message Date
520fb03bfd log: add --use-json-log for JSON logging 2019-07-28 12:05:50 +01:00
5a44bafa4e fstest: add fs.ErrorCantShareDirectories for backends which can only share files 2019-07-24 20:34:29 +01:00
8fdce31700 config: Fix hiding of options from the configurator 2019-07-24 20:34:29 +01:00
8e8b78d7e5 Implement --compare-dest & --copy-dest Fixes #3278 2019-07-22 19:42:29 +01:00
19ae053168 rcserver: remove _async key from input parameters after parsing so later operations won't get confused - fixes #3346 2019-07-20 19:35:10 +02:00
b4266da4eb sync: fix SyncSuffix tests #3272 2019-07-03 17:36:22 +01:00
9cafeeb4b6 dirtree: make tests more reliable 2019-07-02 16:29:40 +01:00
bc70bff125 fs/dirtree: factor DirTree out of fs/walk and add tests 2019-07-02 15:26:55 +01:00
387b496d1e operations: fix tests TestMoveFileBackupDir and TestCopyFileBackupDir again
Commit 734f504d5f wasn't tested properly and had a typo which
caused it not to build :-(
2019-07-02 12:22:29 +01:00
734f504d5f operations: fix tests TestMoveFileBackupDir and TestCopyFileBackupDir
..so they don't run on backends which can't move or copy.
2019-07-02 10:46:49 +01:00
cc0800a72e march: return errors when listing dirs
Partially fixes #3172
2019-07-01 15:32:46 +01:00
70c6b01f54 fs: Higher units for ETA - fixes #3221 2019-07-01 12:09:19 +01:00
7b2b396d37 context: fix errgroup interaction with context
Fixes #3307
2019-07-01 11:51:51 +01:00
de14378734 Implement --suffix without --backup-dir for current dir
Fixes #2801
2019-07-01 10:46:26 +01:00
eea1b6de32 Abstract --Backup-dir checks so can be applied across Sync, Copy, Move 2019-07-01 10:46:26 +01:00
dde4dd0198 fichier: 1fichier support - fixes #2908
This was started by Fionera, finished off by Laura with fixes and more
docs from Nick.

Co-authored-by: Fionera <fionera@fionera.de>
Co-authored-by: Nick Craig-Wood <nick@craig-wood.com>
2019-06-30 18:35:01 +01:00
6ff7b2eaab fs: add fs.ErrorCantUploadEmptyFiles
Any backends which can't upload 0 length files should return this
errror.
2019-06-30 18:11:45 +01:00
38ebdf54be sync/operations: don't use zero length files in tests
We now have a backend (fichier) which doesn't support 0 length
files. Therefore all 0 length files in the tests have been replaced
with length 1.

In a future commit we will implement a test for 0 length files.
2019-06-30 18:11:45 +01:00
276f8cccf6 rc: return current settings if core/bwlimit called without parameters 2019-06-24 13:22:24 +01:00
0ae844d1f8 config: reset environment variables in config file test to fix build 2019-06-22 17:49:23 +01:00
ba72e62b41 fs/config: Add method to reload configfile from disk
Fixes #3268
2019-06-19 14:47:54 +01:00
5935cb0a29 jobs: add ability to stop async jobs
Depends on #3257
2019-06-19 14:17:41 +01:00
f78cd1e043 Add context propagation to rclone
- Change rclone/fs interfaces to accept context.Context
- Update interface implementations to use context.Context
- Change top level usage to propagate context to lover level functions

Context propagation is needed for stopping transfers and passing other
request-scoped values.
2019-06-19 11:59:46 +01:00
6a2a075c14 fs/cache: fix locking
This was causing `fatal error: sync: unlock of unlocked mutex` if a
panic ocurred in fsNewFs.
2019-06-19 10:50:59 +01:00
4549305fec Start v1.48.0-DEV development 2019-06-15 18:32:17 +01:00
245fed513a Version v1.48.0 2019-06-15 13:55:41 +01:00
52332a4b24 moveto: fix detection of same file name to include the root
Fixes problem introduced in d2be792d5e
2019-06-15 13:55:41 +01:00
75606dcc27 sync: fix tests on union remote 2019-06-15 10:53:36 +01:00
f3719fe269 fs/cache: unlock mutex in cache.Get to allow recursive calls
This fixes the test lockup in the union tests
2019-06-15 10:42:53 +01:00
d2be792d5e moveto: fix case-insensitive same remote move 2019-06-15 10:06:01 +01:00
2793d4b4cc remove duplicate code 2019-06-15 10:02:25 +01:00
30ac9d920a enable creating encrypted config through external script invocation - fixes #3127 2019-06-15 10:02:25 +01:00
93207ead9c rc/jobs: make job expiry timeouts configurable 2019-06-15 09:55:32 +01:00
976a020a2f Use rclone.conf from rclone executable directory if already existing 2019-06-12 10:08:00 +01:00
903ede52cd config: make config create/update encrypt passwords where necessary
Before this change when using "rclone config create" it wasn't
possible to add passwords in one go, it was necessary to call "rclone
config password" to add the passwords afterwards as "rclone config
create" didn't obscure passwords.

After this change "rclone config create" and "rclone config update"
will obscure passwords as necessary as will the corresponding API
calls config/create and config/update.

This makes "rclone config password" and its API config/password
obsolete, however they will be left for backwards compatibility.
2019-06-10 18:08:55 +01:00
f681d32996 rc: Fix serving bucket based objects with --rc-serve
Before this change serving bucket based objects
`[remote:bucket]/path/to/object` would fail with 404 not found.

This was because the leading `/` in `/path/to/object` was being passed
to NewObject.
2019-06-10 11:59:06 +01:00
34f03ce590 operations: ignore negative sizes when calculating total (#3135) 2019-06-09 16:00:41 +01:00
4b27c6719b fs: Allow sync of a file and a directory with the same name
When sorting fs.DirEntries we sort by DirEntry type and
when synchronizing files let the directories be before objects,
so when the destintation fs doesn't support duplicate names,
we will only lose duplicated object instead of whole directory.

The enables synchronisation to work with a file and a directory of the same name
which is reasonably common on bucket based remotes.
2019-06-09 15:57:05 +01:00
454dfd3c9e rc: Add operations/fsinfo: Return information about the remote
This returns a information about the remote including Name, Root,
Hashes and optional Features.
2019-06-08 09:19:07 +01:00
e1cf551ded fs: add Features.Enabled to return map of enabled features by name 2019-06-08 08:46:53 +01:00
1aa65d60e1 lsjson: add IsBucket field for bucket based remote listing of the root 2019-06-07 17:28:15 +01:00
ac4c8d8dfc cmd/providers: add DefaultStr, ValueStr and Type fields
These fields are auto generated:
- DefaultStr - a string rendering of Default
- ValueStr - a string rendering of Value
- Type - the type of the option
2019-06-05 16:23:42 +01:00
1124c423ee fs: add --ignore-case-sync for forced case insensitivity - fixes #2773 2019-06-03 21:12:10 +01:00
e7c20e0bce operations: make move and copy individual files obey --backup-dir
Before this change, when using rclone copy or move with --backup-dir
and the source was a single file, rclone would fail to use the backup
directory.

This change looks up the backup directory in the Fs cache and uses it
as appropriate.

This affects any commands which call operations.MoveFile or
operations.CopyFile which includes rclone move/moveto/copy/copyto
where the source is a single file.

Fixes #3219
2019-05-27 16:14:55 +01:00
8ee6034b23 Look for Fs in the cache rather than calling NewFs directly
This will save operations when rclone is used in remote control mode
or with the same remote multiple times in the command line.
2019-05-27 16:14:55 +01:00
206e1caa99 fs/cache: factor Fs caching from fs/rc into its own package 2019-05-27 16:14:55 +01:00
0e23c4542f sync: fix integrations tests
2eb31a4f1d broke the integration tests for remotes which use
Copy+Delete as server side Move.
2019-05-12 09:50:20 +01:00
edda6d91cd Use go-homedir to read the home directory more reliably 2019-05-11 23:39:04 +01:00
7d70e92664 operations: enable multi threaded downloads - Fixes #2252
This implements the --multi-thread-cutoff and --multi-thread-streams
flags to control multi thread downloading to the local backend.
2019-05-11 23:35:19 +01:00
687cbf3ded operations: if --ignore-checksum is in effect, don't calculate checksum
Before this change we calculated the checksum which is potentially
time consuming and then ignored the result.  After the change we don't
calculate the checksum if we are about to ignore it.
2019-05-11 23:35:19 +01:00