Commit Graph

1577 Commits

Author SHA1 Message Date
7b2b396d37 context: fix errgroup interaction with context
Fixes #3307
2019-07-01 11:51:51 +01:00
de14378734 Implement --suffix without --backup-dir for current dir
Fixes #2801
2019-07-01 10:46:26 +01:00
eea1b6de32 Abstract --Backup-dir checks so can be applied across Sync, Copy, Move 2019-07-01 10:46:26 +01:00
dde4dd0198 fichier: 1fichier support - fixes #2908
This was started by Fionera, finished off by Laura with fixes and more
docs from Nick.

Co-authored-by: Fionera <fionera@fionera.de>
Co-authored-by: Nick Craig-Wood <nick@craig-wood.com>
2019-06-30 18:35:01 +01:00
6ff7b2eaab fs: add fs.ErrorCantUploadEmptyFiles
Any backends which can't upload 0 length files should return this
errror.
2019-06-30 18:11:45 +01:00
38ebdf54be sync/operations: don't use zero length files in tests
We now have a backend (fichier) which doesn't support 0 length
files. Therefore all 0 length files in the tests have been replaced
with length 1.

In a future commit we will implement a test for 0 length files.
2019-06-30 18:11:45 +01:00
276f8cccf6 rc: return current settings if core/bwlimit called without parameters 2019-06-24 13:22:24 +01:00
0ae844d1f8 config: reset environment variables in config file test to fix build 2019-06-22 17:49:23 +01:00
ba72e62b41 fs/config: Add method to reload configfile from disk
Fixes #3268
2019-06-19 14:47:54 +01:00
5935cb0a29 jobs: add ability to stop async jobs
Depends on #3257
2019-06-19 14:17:41 +01:00
f78cd1e043 Add context propagation to rclone
- Change rclone/fs interfaces to accept context.Context
- Update interface implementations to use context.Context
- Change top level usage to propagate context to lover level functions

Context propagation is needed for stopping transfers and passing other
request-scoped values.
2019-06-19 11:59:46 +01:00
6a2a075c14 fs/cache: fix locking
This was causing `fatal error: sync: unlock of unlocked mutex` if a
panic ocurred in fsNewFs.
2019-06-19 10:50:59 +01:00
4549305fec Start v1.48.0-DEV development 2019-06-15 18:32:17 +01:00
245fed513a Version v1.48.0 2019-06-15 13:55:41 +01:00
52332a4b24 moveto: fix detection of same file name to include the root
Fixes problem introduced in d2be792d5e
2019-06-15 13:55:41 +01:00
75606dcc27 sync: fix tests on union remote 2019-06-15 10:53:36 +01:00
f3719fe269 fs/cache: unlock mutex in cache.Get to allow recursive calls
This fixes the test lockup in the union tests
2019-06-15 10:42:53 +01:00
d2be792d5e moveto: fix case-insensitive same remote move 2019-06-15 10:06:01 +01:00
2793d4b4cc remove duplicate code 2019-06-15 10:02:25 +01:00
30ac9d920a enable creating encrypted config through external script invocation - fixes #3127 2019-06-15 10:02:25 +01:00
93207ead9c rc/jobs: make job expiry timeouts configurable 2019-06-15 09:55:32 +01:00
976a020a2f Use rclone.conf from rclone executable directory if already existing 2019-06-12 10:08:00 +01:00
903ede52cd config: make config create/update encrypt passwords where necessary
Before this change when using "rclone config create" it wasn't
possible to add passwords in one go, it was necessary to call "rclone
config password" to add the passwords afterwards as "rclone config
create" didn't obscure passwords.

After this change "rclone config create" and "rclone config update"
will obscure passwords as necessary as will the corresponding API
calls config/create and config/update.

This makes "rclone config password" and its API config/password
obsolete, however they will be left for backwards compatibility.
2019-06-10 18:08:55 +01:00
f681d32996 rc: Fix serving bucket based objects with --rc-serve
Before this change serving bucket based objects
`[remote:bucket]/path/to/object` would fail with 404 not found.

This was because the leading `/` in `/path/to/object` was being passed
to NewObject.
2019-06-10 11:59:06 +01:00
34f03ce590 operations: ignore negative sizes when calculating total (#3135) 2019-06-09 16:00:41 +01:00
4b27c6719b fs: Allow sync of a file and a directory with the same name
When sorting fs.DirEntries we sort by DirEntry type and
when synchronizing files let the directories be before objects,
so when the destintation fs doesn't support duplicate names,
we will only lose duplicated object instead of whole directory.

The enables synchronisation to work with a file and a directory of the same name
which is reasonably common on bucket based remotes.
2019-06-09 15:57:05 +01:00
454dfd3c9e rc: Add operations/fsinfo: Return information about the remote
This returns a information about the remote including Name, Root,
Hashes and optional Features.
2019-06-08 09:19:07 +01:00
e1cf551ded fs: add Features.Enabled to return map of enabled features by name 2019-06-08 08:46:53 +01:00
1aa65d60e1 lsjson: add IsBucket field for bucket based remote listing of the root 2019-06-07 17:28:15 +01:00
ac4c8d8dfc cmd/providers: add DefaultStr, ValueStr and Type fields
These fields are auto generated:
- DefaultStr - a string rendering of Default
- ValueStr - a string rendering of Value
- Type - the type of the option
2019-06-05 16:23:42 +01:00
1124c423ee fs: add --ignore-case-sync for forced case insensitivity - fixes #2773 2019-06-03 21:12:10 +01:00
e7c20e0bce operations: make move and copy individual files obey --backup-dir
Before this change, when using rclone copy or move with --backup-dir
and the source was a single file, rclone would fail to use the backup
directory.

This change looks up the backup directory in the Fs cache and uses it
as appropriate.

This affects any commands which call operations.MoveFile or
operations.CopyFile which includes rclone move/moveto/copy/copyto
where the source is a single file.

Fixes #3219
2019-05-27 16:14:55 +01:00
8ee6034b23 Look for Fs in the cache rather than calling NewFs directly
This will save operations when rclone is used in remote control mode
or with the same remote multiple times in the command line.
2019-05-27 16:14:55 +01:00
206e1caa99 fs/cache: factor Fs caching from fs/rc into its own package 2019-05-27 16:14:55 +01:00
0e23c4542f sync: fix integrations tests
2eb31a4f1d broke the integration tests for remotes which use
Copy+Delete as server side Move.
2019-05-12 09:50:20 +01:00
edda6d91cd Use go-homedir to read the home directory more reliably 2019-05-11 23:39:04 +01:00
7d70e92664 operations: enable multi threaded downloads - Fixes #2252
This implements the --multi-thread-cutoff and --multi-thread-streams
flags to control multi thread downloading to the local backend.
2019-05-11 23:35:19 +01:00
687cbf3ded operations: if --ignore-checksum is in effect, don't calculate checksum
Before this change we calculated the checksum which is potentially
time consuming and then ignored the result.  After the change we don't
calculate the checksum if we are about to ignore it.
2019-05-11 23:35:19 +01:00
822483aac5 accounting: enable accounting without passing through the stream #2252
This is in preparation for multithreaded downloads
2019-05-11 23:35:19 +01:00
2eb31a4f1d sync: move Transferring into operations.Copy
This makes the code more consistent with the operations code setting
the transfer statistics up.
2019-05-11 23:35:19 +01:00
0655738da6 operations: re-work reopen framework so it can take a RangeOption #2252
This is in preparation for multipart downloads.
2019-05-11 23:35:19 +01:00
7c4fe3eb75 local: define OpenWriterAt interface and test and implement it #2252
This will enable multipart downloads in future commits
2019-05-11 23:35:19 +01:00
72721f4c8d copyurl: honor --no-check-certificate 2019-05-11 17:44:58 +01:00
0d511b7878 cmd: implement --stats-one-line-date and --stats-one-line-date-format 2019-05-11 17:39:57 +01:00
Jon
bbe28df800 docs: Fix typo: Dump HTTP bodies -> Dump HTTP headers 2019-05-11 16:40:40 +01:00
d04b0b856a fserrors: use errors.Walk for the wrapped error types 2019-05-01 16:56:08 +01:00
577fda059d rc: fix race in tests 2019-05-01 16:09:50 +01:00
9df322e889 tests: make test servers choose a random port to make more reliable
Tests have been randomly failing with messages like

    listen tcp 127.0.0.1:51778: bind: address already in use

Rework all the test servers so they choose a random free port on
startup and use that for the tests to avoid.
2019-05-01 16:09:50 +01:00
48c09608ea fix spelling 2019-04-30 14:12:18 +02:00
a57756a05c lsjson, lsf: support showing the Tier of the object 2019-04-16 13:34:06 +01:00