Commit Graph

80 Commits

Author SHA1 Message Date
Nick Craig-Wood
de35f1c165 Show WARN in integration tests if remote not configured 2018-02-02 09:50:58 +00:00
Nick Craig-Wood
f9806848fe fstest: use the difficult file name for server side copy #2001
This should detect re-occurrence of #315
2018-01-23 09:37:33 +00:00
Nick Craig-Wood
c098e25552 fstest: Skip RangeOption test on Appveyor also 2018-01-22 11:10:29 +00:00
Nick Craig-Wood
ea69deaa4c fstests: Skip RangeOption test in CI until all implemented 2018-01-21 18:09:16 +00:00
Nick Craig-Wood
8653944a6d Make RangeOption manadatory for Open - #1825
Add an integration test to make sure all backends implement
RangeOption correctly.
2018-01-21 17:09:12 +00:00
Nick Craig-Wood
5ad226ab54 fs: Add dir option to fs.Purge #1891
Purge optional interface needs fixing too.
2018-01-19 11:45:50 +00:00
Nick Craig-Wood
11da2a6c9b Break the fs package up into smaller parts.
The purpose of this is to make it easier to maintain and eventually to
allow the rclone backends to be re-used in other projects without
having to use the rclone configuration system.

The new code layout is documented in CONTRIBUTING.
2018-01-15 17:51:14 +00:00
Nick Craig-Wood
87335de8a8 fs: fix filename normalization issues in the tests when running on OS X 2017-09-17 15:31:22 +01:00
Stefan Breunig
4e189fe6e7 fstests: only test uploadswith indeterminate size on remotes that support it 2017-08-22 07:19:43 +02:00
Stefan Breunig
28a18303f3 implement rcat – fixes #230, fixes #1001 2017-08-03 21:42:35 +02:00
Nick Craig-Wood
7ec24ad67a fstests: Use a different container after the Rmdir
Use a new directory here.  This is for the container based remotes
which take time to create and destroy a container (eg azure blob)
2017-08-03 20:01:05 +01:00
Nick Craig-Wood
cd4895690a fstest: Factor test initialisation into Initialise() 2017-07-26 16:38:33 +01:00
Nick Craig-Wood
da2e2544ee Fix tests on Windows 2017-07-08 16:26:41 +01:00
Nick Craig-Wood
6c3bf629a1 yandex: fix fs.Name()
Put in tests for fs.Root() and fs.Name() for all remotes
2017-07-03 13:39:31 +01:00
Nick Craig-Wood
e7d04fc103 Create fs.Directory interface and use it everywhere 2017-06-30 14:26:59 +01:00
Nick Craig-Wood
eaa717b88a Fix crypt obfuscate tests with Windows 2017-06-18 22:53:19 +01:00
Nick Craig-Wood
8a6a8b9623 Change List interface and add ListR optional interface
This simplifies the implementation of remotes.  The only required
interface is now `List` which is a simple one level directory list.

Optionally remotes may implement `ListR` if they have an efficient way
of doing a recursive list.
2017-06-14 16:49:40 +01:00
Nick Craig-Wood
7e20e16cff core: Implement Walk directory listing and use in place of Lister
This is in preparation for removing the Lister code and replacing the
fundamental operation in the Fs with listing a single directory.
2017-06-14 16:49:40 +01:00
Nick Craig-Wood
68333d34a1 dropbox: make setting mod time on existing files work properly
This is a fix left over from the v2 conversion.  Dropbox ignores the
client modification on an incoming file if it was identical to the
existing file.  This change deletes the existing file first before
re-uploading the new one.
2017-06-13 13:58:39 +01:00
Nick Craig-Wood
740b3f6ae2 Fix problems found with ineffassign 2017-06-13 11:52:36 +01:00
Nick Craig-Wood
bdc19b7c8a fstests: fix -remote flag to override test target 2017-06-04 22:38:29 +01:00
Stefan Breunig
a2e3af0523 poll for Google Drive changes when mounted 2017-05-28 17:54:52 +01:00
Nick Craig-Wood
412dacf8be Add a test for partial reads to all remotes 2017-05-19 12:28:47 +01:00
Nick Craig-Wood
79e3c67bbd local, yandex, dropbox: fix NewObject suceeding on a directory #1079
Add tests to make it consistent across all remotes
2017-02-25 11:09:57 +00:00
Nick Craig-Wood
73a96dc588 Improve directory listing tests to detect issue #1165 2017-02-22 11:53:40 +00:00
Nick Craig-Wood
f3c5745468 Add srcRemote and dstRemote parameters to DirMove #954 2017-02-16 17:42:37 +00:00
Nick Craig-Wood
ac1c041377 Redo log level flags
* -vv or --log-level DEBUG
  * -v or --log-level INFO
  * --log-level NOTICE (default)
  * -q --log-level ERROR

Replace Config.Verbose and Config.Quiet with Config.LogLevel

Fixes #739 Fixes #1108 Fixes #1000
2017-02-11 20:22:42 +00:00
Nick Craig-Wood
9fdeb82328 Fix tests under Windows 2017-01-20 17:12:05 +00:00
Nick Craig-Wood
1fa258c2b4 Define a new Features() method for Fs
Optional interfaces are becoming more important in rclone,
--track-renames and --backup-dir both rely on them.

Up to this point rclone has used interface upgrades to define optional
behaviour on Fs objects.  However when one Fs object wraps another it
is very difficult for this scheme to work accurately.  rclone has
relied on specific error messages being returned when the interface
isn't supported - this is unsatisfactory because it means you have to
call the interface to see whether it is supported.

This change enables accurate detection of optional interfaces by use
of a Features struct as returned by an obligatory Fs.Features()
method.  The Features struct contains flags and function pointers
which can be tested against nil to see whether they can be used.

As a result crypt and hubic can accurately reflect the capabilities of
the underlying Fs they are wrapping.
2017-01-16 17:33:25 +00:00
Nick Craig-Wood
f73ee5eade Make all config file variables be settable in the environment
These are set in the form RCLONE_CONFIG_remote_option where remote is
the uppercased remote name and option is the uppercased config file
option name.  Note that RCLONE_CONFIG_remote_TYPE must be set if
defining a new remote.

Fixes #616
2017-01-03 22:42:47 +00:00
Nick Craig-Wood
80c044f2d3 Stop overwriting global remote in tests 2016-12-20 14:15:11 +00:00
Nick Craig-Wood
aaa1370a36 Add directory parameter to Rmdir and Mkdir #100 #831
This will enable rclone to manage directories properly in the future.
2016-11-26 12:02:53 +00:00
Stefan Breunig
77f38cb6f1 acd: extend move test to check conflict cases for two step rename+move 2016-11-11 18:05:24 +00:00
Nick Craig-Wood
441951a93b Stop removing failed upload to cloud storage remotes - fixes #559
We do remove a partially written file on local so we don't have
corrupted files lying around.
2016-11-04 21:34:25 +00:00
Nick Craig-Wood
aef2ac5c04 Add options for Open and implement Range for all remotes 2016-10-05 21:03:56 +01:00
Nick Craig-Wood
945f49ab5e Make ContentType be preserved for cloud -> cloud copies - fixes #733 2016-10-03 20:02:04 +01:00
Nick Craig-Wood
c2599cb116 Fix crypt tests on Windows 2016-08-24 22:21:34 +01:00
Nick Craig-Wood
226c2a0d83 Implement crypt for encrypted remotes - #219 2016-08-23 17:45:37 +01:00
Nick Craig-Wood
b4b4b6cb1c Allow Fs tests to declare new config items 2016-08-23 17:45:37 +01:00
Nick Craig-Wood
326dcf2470 Add more troublesome symbols to test cases
These are from #623 #620 #218
2016-08-23 14:28:05 +01:00
Nick Craig-Wood
4f9e805d44 acd: Work around 408 REQUEST_TIMEOUT and 504 GATEWAY_TIMEOUT errors
Amazon Drive sometimes returns errors at the end of large uploads

  * 408 REQUEST_TIMEOUT
  * 504 GATEWAY_TIMEOUT
  * 500 Internal error

The file may have been uploaded correctly though, so, on error, wait
for up to 2 minutes for it to appear if it was fully
uploaded (configure timeout with --acd-upload-wait-time).

Issues: #601 #605 #606
2016-08-18 17:39:23 +01:00
Nick Craig-Wood
ac3c83f966 Fix integration tests for drive 2016-07-12 21:38:15 +01:00
Nick Craig-Wood
a6056408dd Fix move command - stop it running for overlapping fses - fixes #577
* Make move command check for overlapping remotes and refuse to run
  * Do copy/delete rather than all the copies then all the deletes
  * Doesn't purge the source - this was unexpected behaviour see #512 and #416
  * Add -list-retries flag to test suite to control retries

This changes the semantics of `move` slightly.  However it now errs on
the side of not deleting stuff.
2016-07-12 10:49:37 +01:00
Nick Craig-Wood
28f4061892 Add two more classes of error Fatal and NoRetry
These are for remotes to signal that they have a fatal error and don't
want to continue (eg cap exceeded) or that a particular file shouldn't
be retried for some reason.
2016-07-04 13:45:52 +01:00
Nick Craig-Wood
70dc97231e Convert more tests to use assert/require 2016-06-30 15:45:30 +01:00
Nick Craig-Wood
ab43005422 Make NewObject return an error
* make it return an error
  * make a canonical error fs.ErrorNotFound
  * make a test for it
  * remove logs/debugs of error
2016-06-28 08:51:57 +01:00
Nick Craig-Wood
b1f131964e Rename NewFsObject to NewObject 2016-06-28 08:51:57 +01:00
Nick Craig-Wood
1a87b69376 Get rid of LimitedFs - FIXME needs docs on copying single files
If remote:path points to a file make NewFs return a sentinel error
fs.ErrorIsFile and an Fs which points to the parent.

Use this to remove the LimitedFs and just add this file to the
--files-from list.

This means that server side operations can be used also.

Fixes #518
Fixes #545
2016-06-28 08:51:43 +01:00
Nick Craig-Wood
df1092ef33 Change Fs.Put so that it must cope with existing files
This should fix duplicate files on drive and 409 errors on
amazonclouddrive however it will slow down the upload slightly as
another roundtrip will be needed.

None of the other Fses needed adjusting.

Fixes #483
2016-06-13 19:29:10 +01:00
Nick Craig-Wood
67d0375b98 Audit use of log.Print and change to Debug, Log, or ErrorLog as appropriate 2016-06-06 21:23:54 +01:00