Nick Craig-Wood
abc736df1d
cat: Use RangeOption for limited fetches to make more efficient #1825
2018-02-19 18:00:39 +00:00
Nick Craig-Wood
ab0d06eb16
fs: Make RangeOption mandatory #1825
2018-02-19 18:00:39 +00:00
Mateusz
afc963ed92
config: retry saving the config after failure - fixes #2060
2018-02-19 17:59:27 +00:00
Fabian Möller
451cd6d971
fs: add ChunkedReader
2018-02-19 15:45:24 +00:00
Fabian Möller
a647c54888
fs: add RangeSeeker interface
2018-02-19 15:45:24 +00:00
Nick Craig-Wood
a5c3bcc9c7
fshttp: fix idle timeouts for HTTP connections #2057
...
Now we only nudge on the idle timeout after a successful Read or Write
which returns some bytes and no errors.
2018-02-16 10:35:41 +00:00
Bjørn Erik Pedersen
ab8c0a81fa
Add a delete threshold to sync (--max-delete)
...
Fixes #959
2018-02-12 11:29:58 +00:00
Nick Craig-Wood
d0d6b83a7a
fs/accounting: rework to enable accounting to work with crypt and b2
...
This removes the old system of part accounting and replaces it with a
system of popping off the accounting reader and wrapping up new ones
as necessary.
This makes it much easier to carry the context down the chain of
wrapped readers and get the limiting as near as possible to the
output. This makes the accounting more accurate and the bandwidth
limiting smoother.
Fixes #2029 and Fixes #1443
2018-02-02 15:14:41 +00:00
Nick Craig-Wood
bea02fcf52
fs/accounting: factor into separate files without changing functionality
2018-02-02 15:14:40 +00:00
Nick Craig-Wood
6fb868e00c
config: fix --log-level flag after code reorganization - fixes #2043
2018-02-02 14:07:44 +00:00
Nick Craig-Wood
8df78f2b6d
operations: ignore size of objects when they are < 0 #320
...
This allows google docs to be transferred and checked correctly.
2018-01-31 16:22:05 +00:00
Nick Craig-Wood
2eb5cfb7ad
fs: Formalize the ObjectUnWrapper interface
2018-01-31 16:21:41 +00:00
Nick Craig-Wood
fe52502f19
fs: Adjust RangeOption.Decode to return -1 for read to end
...
A Range request can never request 0 bytes however this change was made
to make a clearer signal that the limit means read to the end.
Add test and more documentation and fixup uses
2018-01-27 14:31:29 +00:00
Nick Craig-Wood
9a73688e3a
fs: Add ParseRangeOption to parse incoming Range: requests
2018-01-27 13:16:37 +00:00
Nick Craig-Wood
bc3ee977f4
fs/hash: move interface assertion to tests so it doesn't pull in spf13/flag
2018-01-26 14:35:18 +00:00
Nick Craig-Wood
a6833b68ca
local: factor RangeOption code to Decode() method and readers.LimitedReadCloser #1825
2018-01-22 19:44:00 +00:00
Nick Craig-Wood
d876392d15
onedrive: Factor code into fs.FixRangeOption
2018-01-22 17:05:00 +00:00
Nick Craig-Wood
84bc4dc142
Clarify RangeOption semantics
2018-01-21 09:51:28 +00:00
Nick Craig-Wood
ee66419a27
fs/fserrors: Add test for error from #1964
2018-01-19 17:07:40 +00:00
Nick Craig-Wood
5ad226ab54
fs: Add dir option to fs.Purge #1891
...
Purge optional interface needs fixing too.
2018-01-19 11:45:50 +00:00
Nick Craig-Wood
97c414f025
config/hash: rename more symbols after factoring into own package
2018-01-18 20:27:52 +00:00
Nick Craig-Wood
71722b5b95
config: factor Obscure and Reveal into its own package
2018-01-18 20:19:55 +00:00
Ernest Borowski
5cce74d630
flags: remove --no-traverse flag because it is obsolete - fixes #1813
...
Signed-off-by: Ernest Borowski <er.borowski@gmail.com>
2018-01-18 11:00:25 +00:00
Iakov Davydov
acd55a8f65
local, fs: --exclude-if-present ignores directories which it doesn't have permission for - fixes #1959
2018-01-16 20:00:16 +00:00
Nick Craig-Wood
11da2a6c9b
Break the fs package up into smaller parts.
...
The purpose of this is to make it easier to maintain and eventually to
allow the rclone backends to be re-used in other projects without
having to use the rclone configuration system.
The new code layout is documented in CONTRIBUTING.
2018-01-15 17:51:14 +00:00
Nick Craig-Wood
b8b620f5c2
Move all backends into backend directory
2018-01-12 17:07:38 +00:00
Nick Craig-Wood
712e6a8085
lsf: fix integration tests
2018-01-11 13:52:15 +00:00
Will Gunn
5c7e8d5a2b
fs: Add --stats-file-name-length to specify the printed file name length for stats
...
Fixes #1206
2018-01-10 20:32:36 +00:00
Nick Craig-Wood
c74c3b37da
lsf: add option to print hashes
2018-01-06 17:53:37 +00:00
Nick Craig-Wood
7c71ee1a5b
fs: fix TestListFormat on remotes which return 0 as dir size not -1
2018-01-06 17:47:42 +00:00
Jakub Tasiemski
0d041602cf
cmd: new command lsf
2018-01-06 14:39:31 +00:00
Nick Craig-Wood
b412c745a1
Start v1.39-DEV development
2017-12-23 13:40:28 +00:00
Nick Craig-Wood
f34a9116d4
Version v1.39
2017-12-23 13:07:45 +00:00
Nick Craig-Wood
7e585cda96
fs: fix TestRmdirsLeaveRoot test
2017-12-14 08:57:28 +00:00
lewapm
9c242edc10
rmdirs: add --leave-root flag - fixes #1874
2017-12-13 10:23:54 +00:00
ishuah
b05e472d2e
stats: condensed transfer output to fit 80x25 terminals
2017-12-09 10:48:36 +03:00
remusb
25b073c767
fs: add Wrap feature for FS to identify their parent FS ( #1884 )
2017-12-06 17:14:34 +02:00
Nick Craig-Wood
fa4a25a73b
fs: only test one level of cache
...
Can't test multiple caches at once as can only have 1 DB open at once
2017-12-04 15:50:59 +00:00
Ernest Borowski
934df67aef
filter: warn the user if he use --include and --exclude together fixes #1764
...
Signed-off-by: Ernest Borowski <er.borowski@gmail.com>
2017-12-04 14:20:01 +00:00
Nick Craig-Wood
4b185355df
fs: rcat - use in memory object and Copy for more reliable transfers
2017-12-03 12:14:15 +00:00
Nick Craig-Wood
7d15c33e42
fs: make Copy and Move return the destination object if possible
2017-12-03 12:14:15 +00:00
Nick Craig-Wood
11332a19a0
fs: make an in memory object for short transfers
2017-12-03 12:14:15 +00:00
Nick Craig-Wood
639e812789
fs: integration tests: add SUMMARY heading for log scraping
2017-11-29 15:55:37 +00:00
Nick Craig-Wood
1c6cad2252
fs: integration tests: add 30 minute timeout per test
2017-11-29 13:51:17 +00:00
ishuah
aab8051f50
move: add --delete-empty-src-dirs flag - fixes #1854
2017-11-28 11:38:19 +03:00
Nick Craig-Wood
fdb01437d8
fs: Allow the http Transport to have an optional filter request function
2017-11-24 09:07:56 +00:00
Nick Craig-Wood
02ffd43572
fs: Save the config before asking for a token - fixes #1220
...
Before this if the client_id/client_secret was edited it would
disappear when asking for the new token.
This means the post config is done after the user has confirmed the
config is OK which can't be helped.
2017-11-23 14:01:32 +00:00
Nick Craig-Wood
e53892f53b
fs,drive,dropbox: Make and use new RepeatableReader variants to lower memory use
...
RepeatableReaderSized has a pre-allocated buffer which should help
with memory usage - before it grew the buffer. Since we know the size
of the chunks, pre-allocating it should be much more efficient.
RepeatableReaderBuffer uses the buffer passed in.
RepeatableLimit* are convenience funcitions for wrapping a reader in
an io.LimitReader and then a RepeatableReader with the same buffer
size.
2017-11-23 13:53:46 +00:00
ishuah
6c62fced60
move: fixed root source directories getting deleted after move - fixes #1849
2017-11-23 12:01:35 +03:00
Nick Craig-Wood
159fce0106
fs: fix --cache-dir to have some effect
2017-11-22 17:05:02 +00:00
