rclone/notes.txt

Names
  * rclone - domain ok, apt ok

Todo
  * Make a test suite which can run on all the given types of fs
  * Copy should use the sync code as it is more efficient at directory listing
  * Factor fses into own packages
  * FIXME: ls without an argument for buckets/containers?
  * FIXME: More -dry-run checks for object transfer
  * Might be quicker to check md5sums first? for swift <-> swift certainly, and maybe for small files
  * Ignoring the pseudo directories
    * if object.PseudoDirectory {
      * fmt.Printf("%9s %19s %s\n", "Directory", "-", fs.Remote())
  * Make Account wrapper
    * limit bandwidth for a pool of all individual connectinos
    * do timeouts by setting a limit, seeing whether io has happened
      and resetting it if it has
    * make Account do progress meter
  * Make logging controllable with flags (mostly done)
  * -timeout: Make all timeouts be settable with command line parameters
  * Check the locking in swift module!
  * Windows paths? Do we need to translate / and \?
  * Make a fs.Errorf and count errors and log them at a different level
  * Add max object size to fs metadata - 5GB for swift, infinite for local, ? for s3
    * tie into -max-size flag

Drive
  * Do we need the secrets or just the code?  If just the code then
    can make a web service which does the request on the clients
    behalf so don't need to expose the client secrets
  * Apparently we don't need -drive-client-id or -drive-client-secret once we have a token
  * Make a cgi which we send the user to
    * It has the client secrets
    * It gets google to authenticate
    * It receives the token back
    * It displays the token to the user to paste in to the code
    * Should be https really
  * Sometimes get: Failed to copy: Upload failed: googleapi: Error 403: Rate Limit Exceeded
    * quota is 100.0 requests/second/user

Ideas
  * could do encryption - put IV into metadata?
  * optimise remote copy container to another container using remote
    copy if local is same as remote - use an optional Copier interface
  * Allow subpaths container:/sub/path
  * look at auth from env in s3 module - add to swift?
  * support
      * sftp
      * scp
      * Google cloud storage: https://developers.google.com/storage/
      * Google drive: https://developers.google.com/drive/
      * rsync over ssh
      * dropbox: https://github.com/nickoneill/go-dropbox (no MD5s)
  * grive seems to have its secrets in the source code which would make things easier!

     

Need to make directory objects otherwise can't upload an empty directory
  * Or could upload empty directories only?
  * Can't purge a local filesystem because it leaves the directories behind

s3
  * Can maybe set last modified?
    * https://forums.aws.amazon.com/message.jspa?messageID=214062
    * Otherwise can set metadata
  * Returns etag and last modified in bucket list


Bugs

local & drive need to delete directories

2013/01/18 16:31:32 Waiting for deletions to finish
2013/01/18 16:31:32 z3: FIXME Skipping directory
2013/01/18 16:31:32 z3/x: Deleted
2013/01/18 16:31:32 Deleting path
2013/01/18 16:31:32 Rmdir failed: remove z3: directory not empty