Commit Graph

85 Commits

Author SHA1 Message Date
Nick Craig-Wood
8ef551bf9c Make dedupe remove identical copies without asking and add non interactive mode - fixes #338
* Now removes identical copies without asking
  * Now obeys `--dry-run`
  * Implement `--dedupe-mode` for non interactive running
    * `--dedupe-mode interactive` - interactive the default.
    * `--dedupe-mode skip` - removes identical files then skips anything left.
    * `--dedupe-mode first` - removes identical files then keeps the first one.
    * `--dedupe-mode newest` - removes identical files then keeps the newest one.
    * `--dedupe-mode oldest` - removes identical files then keeps the oldest one.
    * `--dedupe-mode rename` - removes identical files then renames the rest to be different.
  * Add tests which will only run on Google Drive.
2016-03-06 18:15:01 +00:00
Nick Craig-Wood
229ea3f86c Stop --update tests running on remotes which don't do mod time 2016-03-01 07:26:33 +00:00
Nick Craig-Wood
280ac26464 Implement -u/--update so creation times can be used on all remotes - #226 2016-02-29 17:46:40 +00:00
Nick Craig-Wood
867a26fe4f Implement --low-level-retries flag - fixes #266 2016-02-25 22:58:21 +00:00
Nick Craig-Wood
d2219a800a Fix and document the move command - fixes #334
* Don't attempt to use server side Move unless they are on the same Fs
  * Fix move in the presense of filters
2016-02-25 20:05:34 +00:00
klauspost
ef06371c93 Create separate interface for object information.
Take out read-only information about a Fs in a separate struct to limit access.

See discussion at #282.
2016-02-19 13:31:09 +00:00
klauspost
bfd7601cf9 Add configuration file encryption
See #317 for details.

Use `rclone config` to add/change/remove password.

Tests that loads the default configuration will now fail with a better error message, and add a switch that makes it possible to disable password prompts and fail instead.

Make it possible to use the "RCLONE_CONFIG_PASS" environment variable as password for configuration.
2016-02-16 16:32:05 +01:00
Nick Craig-Wood
1373efaa39 Delete command which does obey the filters - fixes #327 2016-01-31 16:06:04 +00:00
Nick Craig-Wood
74a71f7824 Add tests for --delete-before, --delete-during and --delete-after 2016-01-26 16:57:09 +00:00
Nick Craig-Wood
18ebec8276 Check remote is empty between integration tests 2016-01-24 12:37:19 +00:00
Nick Craig-Wood
eda4130703 Fix integration tests so they can be run independently and out of order - fixes #291
* Make all integration tests start with an empty remote
  * Add an -individual flag so this can be a different bucket/container/directory
  * Fix up tests after changing the hashers
  * Add sha1sum test
  * Make directory checking in tests sleep more to fix acd inconsistencies
  * Factor integration tests to make more maintainable
  * Ensure remote writes have a fstest.CheckItems() before use
    * this fixes eventual consistency on the directory listings later
  * Call fs.Stats.ResetCounters() before every fs.Sync()

Note that the tests shouldn't be run concurrently as fs.Config is global state.
2016-01-23 17:02:09 +00:00
Nick Craig-Wood
e8daab7971 Fix integration tests for remotes with unsupported hash schemes 2016-01-16 09:45:15 +00:00
klauspost
78c3a5ccfa Add support for multiple hash types.
Add support for multiple hash types with negotiation of common hash types for comparison.

Manually rebased version of #277 (see discussion there)
2016-01-11 13:39:33 +01:00
klauspost
25f22ec561 Add "--ignore-existing" flag.
Add option to completely ignore existing files and not consider them for transfer.

Fixes #274
2016-01-08 08:20:04 +00:00
Nick Craig-Wood
3423de65fa Make canonical place for all fs in fs/all/all.go 2016-01-03 14:12:45 +00:00
dibu28
3ac4407b88 Implement Yandex storage backend - fixes #234 2015-12-30 12:11:46 +00:00
Nick Craig-Wood
33b3eea6ec Implement Backblaze B2 - fixes #224 2015-12-30 10:05:07 +00:00
Nick Craig-Wood
113624691a Add -dump-headers and -dump-bodies flags for operations test debugging 2015-12-30 09:35:35 +00:00
Nick Craig-Wood
fcea3777c0 Implement Hubic storage system - fixes #200 2015-11-14 08:08:52 +00:00
Nick Craig-Wood
80ffbade22 Fix deletion of some excluded files without --delete-excluded #205
This only happened if the destination file was present but the source
file was missing.
2015-11-12 11:46:04 +00:00
Nick Craig-Wood
2fcb8f5db7 Add support for Microsoft One Drive - fixes #10
* Still to do
    * Copy
    * Move
    * MoveDir
2015-11-07 10:19:10 +00:00
Nick Craig-Wood
177dbbc29a Implement rclone size for measuring remotes - fixes #153 2015-10-03 16:04:21 +01:00
Nick Craig-Wood
a91bcaaeb0 Implement rsync like include and exclude - fixes #27
* Implement include/exclude
  * Implement rsync compatible file globbing
  * Implement command line filtering flags
    * --delete-excluded - Delete files on dest excluded from sync
    * --filter - Add a file-filtering rule
    * --filter-from - Read filtering patterns from a file
    * --exclude - Exclude files matching pattern
    * --exclude-from - Read exclude patterns from file
    * --include - Include files matching pattern
    * --include-from - Read include patterns from file
    * --files-from - Read list of source-file nam
    * --min-size - Don't transfer any file smaller than this in k or suffix k|M|G
    * --max-size - Don't transfer any file larger than this in k or suffix k|M|G
  * Document
2015-09-28 19:18:21 +01:00
Nick Craig-Wood
5f20ae707d Make lsl output times in localtime and fix tests - fixes #141 2015-09-22 19:04:12 +01:00
Nick Craig-Wood
fec16b0ac8 acd: Skip test on FS which don't support ModifiedTime 2015-09-09 23:21:50 +01:00
Nick Craig-Wood
8c3df224ef Implement Amazon Cloud Drive - fixes #45
* Optional interfaces Copier, Mover, DirMover not done
2015-09-08 21:02:48 +01:00
Nick Craig-Wood
59ba8f28c8 Implement move command - fixes #35
* Define Mover interface to move a single object
  * Define DirMover interface to move a directory
  * Implement DirMove operation
  * Add `rclone move` command
  * Tests for Dir Move

To Do
  * Implement Move, DirMover in local, drive, dropbox
  * unit test for Mover
  * unit test for DirMover
2015-08-28 08:49:16 +01:00
Nick Craig-Wood
a96b522958 Implement server side copies if possible - fixes #99
Add optional fs.Copier interface

Implemented for
  * swift
  * s3
  * drive
  * dropbox
  * google cloud storage
2015-08-23 21:18:38 +01:00
Nick Craig-Wood
1d9e76bb0f dropbox: remove datastore - Fixes #55 #84
This means that dropbox no longer stores MD5SUMs and modified times.

Fixup the tests so that blank MD5SUMs are ignored, and that if
Precision is set to a fs.ModTimeNotSupported, ModTimes can be ignored too.

This opens the door for other FSs which don't support metadata easily.
2015-08-17 17:26:36 +01:00
Leonid Shalupov
bd5f685d0a fix TestLsLong on non-UTC timezones 2015-06-14 15:23:56 +01:00
Nick Craig-Wood
e22b445cff Implement --size-only flag to sync on size not checksum & modtime - fixes #75 2015-06-06 08:49:01 +01:00
Alex Couper
7af1282375 Add --checksum flag to only discard transfers by MD5SUM - fixes #61
Useful for copying between backends where checksum fetching is fast,
ie any of s3, swift, drive or googlecloudstorage.
2015-06-05 19:46:03 +01:00
Nick Craig-Wood
80000b904c Version v1.11 2015-03-04 17:59:31 +00:00
Nick Craig-Wood
b4e19cfd62 windows: make tests work properly 2015-02-07 22:32:51 +00:00
Nick Craig-Wood
18439cf2d7 Move rclonetest into go tests for fs module 2014-08-03 11:18:25 +01:00