drive: Rework token aquisition into config framework and store token in config file

* Fix empty config configuration
  * Alter menus when no remotes
  * Save config file after delete remote

README.md

@@ -1,16 +1,28 @@
 Rclone
 ======
 
+[![Logo](http://rclone.org/rclone-120x120.png)](http://rclone.org/)
+
 Sync files and directories to and from
 
-  * Openstack Swift
-  * Rackspace cloud files
-  * Amazon S3
   * Google Drive
+  * Amazon S3
+  * Openstack Swift / Rackspace cloud files / Memset Memstore
   * The local filesystem
 
-FIXME
+Features
 
+  * MD5SUMs checked at all times for file integrity
+  * Timestamps preserved on files
+  * Partial syncs supported on a whole file basis
+  * Copy mode to just copy new/changed files
+  * Sync mode to make a directory identical
+  * Check mode to check all MD5SUMs
+  * Can sync to and from network, eg two different Drive accounts
+
+Home page
+
+  * http://rclone.org/
 
 Install
 -------
@@ -19,19 +31,269 @@ Rclone is a Go program and comes as a single binary file.
 
 Download the relevant binary from
 
-- http://www.craig-wood.com/nick/pub/rclone/
+  * http://www.craig-wood.com/nick/pub/rclone/
 
 Or alternatively if you have Go installed use
 
     go get github.com/ncw/rclone
 
-and this will build the binary in `$GOPATH/bin`.  You can then modify
+and this will build the binary in `$GOPATH/bin`.
-the source and submit patches.
+
+You can then modify the source and submit patches.
+
+Configure
+---------
+
+First you'll need to configure rclone.  As the object storage systems
+have quite complicated authentication these are kept in a config file
+`.rclone.conf` in your home directory by default.  (You can use the
+-config option to choose a different config file.)
+
+The easiest way to make the config is to run rclone with the config
+option, Eg
+
+    rclone config
+
+Here is an example of making an s3 configuration
+
+```
+$ rclone config
+No remotes found - make a new one
+n) New remote
+q) Quit config
+n/q> n
+name> remote
+What type of source is it?
+Choose a number from below
+ 1) swift
+ 2) s3
+ 3) local
+ 4) drive
+type> 2
+AWS Access Key ID.
+access_key_id> accesskey
+AWS Secret Access Key (password). 
+secret_access_key> secretaccesskey
+Endpoint for S3 API.
+Choose a number from below, or type in your own value
+ * The default endpoint - a good choice if you are unsure.
+ * US Region, Northern Virginia or Pacific Northwest.
+ * Leave location constraint empty.
+ 1) https://s3.amazonaws.com/
+ * US Region, Northern Virginia only.
+ * Leave location constraint empty.
+ 2) https://s3-external-1.amazonaws.com
+[snip]
+ * South America (Sao Paulo) Region
+ * Needs location constraint sa-east-1.
+ 9) https://s3-sa-east-1.amazonaws.com
+endpoint> 1
+Location constraint - must be set to match the Endpoint.
+Choose a number from below, or type in your own value
+ * Empty for US Region, Northern Virginia or Pacific Northwest.
+ 1) 
+ * US West (Oregon) Region.
+ 2) us-west-2
+[snip]
+ * South America (Sao Paulo) Region.
+ 9) sa-east-1
+location_constraint> 1
+--------------------
+[remote]
+access_key_id = accesskey
+secret_access_key = secretaccesskey
+endpoint = https://s3.amazonaws.com/
+location_constraint = 
+--------------------
+y) Yes this is OK
+e) Edit this remote
+d) Delete this remote
+y/e/d> y
+Current remotes:
+
+Name                 Type
+====                 ====
+remote               s3
+
+e) Edit existing remote
+n) New remote
+d) Delete remote
+q) Quit config
+e/n/d/q> q
+```
+
+This can now be used like this
+
+```
+rclone lsd remote:// - see all buckets/containers
+rclone ls remote:// - list a bucket
+rclone sync /home/local/directory remote://bucket
+```
+
+See the next section for more details.
 
 Usage
 -----
 
-FIXME
+Rclone syncs a directory tree from local to remote.
+
+Its basic syntax is like this
+
+    Syntax: [options] subcommand <parameters> <parameters...>
+
+See below for how to specify the source and destination paths.
+
+Subcommands
+-----------
+
+    rclone copy source://path dest://path
+
+Copy the source to the destination.  Doesn't transfer
+unchanged files, testing first by modification time then by
+MD5SUM.  Doesn't delete files from the destination.
+
+    rclone sync source://path dest://path
+
+Sync the source to the destination.  Doesn't transfer
+unchanged files, testing first by modification time then by
+MD5SUM.  Deletes any files that exist in source that don't
+exist in destination. Since this can cause data loss, test
+first with the -dry-run flag.
+
+    rclone ls [remote://path]
+
+List all the objects in the the path.
+
+    rclone lsd [remote://path]
+
+List all directoryes/objects/buckets in the the path.
+
+    rclone mkdir remote://path
+
+Make the path if it doesn't already exist
+
+    rclone rmdir remote://path
+
+Remove the path.  Note that you can't remove a path with
+objects in it, use purge for that.
+
+    rclone purge remote://path
+
+Remove the path and all of its contents.
+
+    rclone check source://path dest://path
+
+Checks the files in the source and destination match.  It
+compares sizes and MD5SUMs and prints a report of files which
+don't match.  It doesn't alter the source or destination.
+
+General options:
+  * `-config` Location of the config file
+  * `-transfers=4`: Number of file transfers to run in parallel.
+  * `-checkers=8`: Number of MD5SUM checkers to run in parallel.
+  * `-dry-run=false`: Do a trial run with no permanent changes
+  * `-modify-window=1ns`: Max time difference to be considered the same - this is automatically set usually
+  * `-quiet=false`: Print as little stuff as possible
+  * `-stats=1m0s`: Interval to print stats
+  * `-verbose=false`: Print lots more stuff
+
+Developer options:
+  * `-cpuprofile=""`: Write cpu profile to file
+
+Local Filesystem
+----------------
+
+Paths are specified as normal filesystem paths, so
+
+    rclone sync /home/source /tmp/destination
+
+Will sync source to destination
+
+Swift / Rackspace cloudfiles / Memset Memstore
+----------------------------------------------
+
+Paths are specified as remote://container (or remote:// for the `lsd`
+command.)
+
+So to copy a local directory to a swift container called backup:
+
+    rclone sync /home/source swift://backup
+
+The modified time is stored as metadata on the object as
+`X-Object-Meta-Mtime` as floating point since the epoch.
+
+This is a defacto standard (used in the official python-swiftclient
+amongst others) for storing the modification time (as read using
+os.Stat) for an object.
+
+Amazon S3
+---------
+
+Paths are specified as remote://bucket
+
+So to copy a local directory to a s3 container called backup
+
+    rclone sync /home/source s3://backup
+
+The modified time is stored as metadata on the object as
+`X-Amz-Meta-Mtime` as floating point since the epoch.
+
+Google drive
+------------
+
+Paths are specified as drive://path  Drive paths may be as deep as required.
+
+The initial setup for drive involves getting a token from Google drive
+which you need to do in your browser.  The `rclone config` walks you
+through it.
+
+Here is an example of how to make a remote called `drv`
+
+```
+$ ./rclone config
+n) New remote
+d) Delete remote
+q) Quit config
+e/n/d/q> n
+name> drv
+What type of source is it?
+Choose a number from below
+ 1) swift
+ 2) s3
+ 3) local
+ 4) drive
+type> 4
+Google Application Client Id - leave blank to use rclone's.
+client_id> 
+Google Application Client Secret - leave blank to use rclone's.
+client_secret> 
+Remote config
+Go to the following link in your browser
+https://accounts.google.com/o/oauth2/auth?access_type=&approval_prompt=&client_id=XXXXXXXXXXXX.apps.googleusercontent.com&redirect_uri=urn%3XXXXX%3Awg%3Aoauth%3XX.0%3Aoob&response_type=code&scope=https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fdrive&state=state
+Log in, then type paste the token that is returned in the browser here
+Enter verification code> X/XXXXXXXXXXXXXXXXXX-XXXXXXXXX.XXXXXXXXX-XXXXX_XXXXXXX_XXXXXXX
+--------------------
+[drv]
+client_id = 
+client_secret = 
+token = {"AccessToken":"xxxx.xxxxxxx_xxxxxxxxxxx_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx","RefreshToken":"1/xxxxxxxxxxxxxxxx_xxxxxxxxxxxxxxxxxxxxxxxxxx","Expiry":"2014-03-16T13:57:58.955387075Z","Extra":null}
+--------------------
+y) Yes this is OK
+e) Edit this remote
+d) Delete this remote
+y/e/d> y
+```
+
+You can then use it like this
+
+    rclone lsd drv://
+    rclone ls drv://
+
+To copy a local directory to a drive directory called backup
+
+    rclone copy /home/source drv://backup
+
+Google drive stores modification times accurate to 1 ms.
 
 License
 -------
@@ -39,21 +301,30 @@ License
 This is free software under the terms of MIT the license (check the
 COPYING file included in this package).
 
+Bugs
+----
+
+  * Doesn't sync individual files yet, only directories.
+  * Drive: Sometimes get: Failed to copy: Upload failed: googleapi: Error 403: Rate Limit Exceeded
+    * quota is 100.0 requests/second/user
+  * Empty directories left behind with Local and Drive
+    * eg purging a local directory with subdirectories doesn't work
+
 Contact and support
 -------------------
 
 The project website is at:
 
-- https://github.com/ncw/rclone
+  * https://github.com/ncw/rclone
 
 There you can file bug reports, ask for help or contribute patches.
 
 Authors
 -------
 
-- Nick Craig-Wood <nick@craig-wood.com>
+  * Nick Craig-Wood <nick@craig-wood.com>
 
 Contributors
 ------------
 
-- Your name goes here!
+  * Your name goes here!

cross-compile

@@ -0,0 +1,28 @@
+#!/bin/sh
+
+# This uses gox from https://github.com/mitchellh/gox
+# Make sure you've run gox -build-toolchain
+
+rm -rf build
+
+gox -output "build/{{.OS}}/{{.Arch}}/{{.Dir}}"
+
+cat <<'#EOF' > build/README.txt
+This directory contains builds of the rclone program.
+
+Rclone a program designed to stress test your disks and find
+failures in them.
+
+Use it to soak test your new disks / memory cards / USB sticks before
+trusting your valuable data to it.
+
+See the project website here: https://github.com/ncw/rclone for
+more details.
+
+The files in this directory are organised by OS and processor type
+
+#EOF
+
+mv build/darwin build/osx
+
+( cd build ; tree . >> README.txt )

drive/drive.go

@@ -9,12 +9,6 @@ package drive
 // FIXME list directory should list to channel for concurrency not
 // append to array
 
-// FIXME perhaps have a drive setup mode where we ask for all the
-// params interactively and store them all in one file
-// - don't need to store client* apparently
-
-// NB permissions of token file is too open
-
 // FIXME need to deal with some corner cases
 // * multiple files with the same name
 // * files can be in multiple directories
@@ -22,14 +16,13 @@ package drive
 // * files with / in name
 
 import (
-	"errors"
+	"encoding/json"
 	"flag"
 	"fmt"
 	"io"
 	"log"
 	"mime"
 	"net/http"
-	"os"
 	"path"
 	"strings"
 	"sync"
@@ -40,36 +33,106 @@ import (
 	"github.com/ncw/rclone/fs"
 )
 
+// Constants
+const (
+	rcloneClientId     = "202264815644.apps.googleusercontent.com"
+	rcloneClientSecret = "X4Z3ca8xfWDb1Voo-F9a7ZxJ"
+	driveFolderType    = "application/vnd.google-apps.folder"
+)
+
+// Globals
+var (
+	// Flags
+	driveFullList = flag.Bool("drive-full-list", true, "Use a full listing for directory list. More data but usually quicker.")
+)
+
 // Register with Fs
 func init() {
 	fs.Register(&fs.FsInfo{
-		Name:  "drive",
+		Name:   "drive",
-		NewFs: NewFs,
+		NewFs:  NewFs,
+		Config: Config,
 		Options: []fs.Option{{
 			Name: "client_id",
-			Help: "Google Application Client Id.",
+			Help: "Google Application Client Id - leave blank to use rclone's.",
-			Examples: []fs.OptionExample{{
-				Value: "202264815644.apps.googleusercontent.com",
-				Help:  "rclone's client id - use this or your own if you want",
-			}},
 		}, {
 			Name: "client_secret",
-			Help: "Google Application Client Secret.",
+			Help: "Google Application Client Secret - leave blank to use rclone's.",
-			Examples: []fs.OptionExample{{
-				Value: "X4Z3ca8xfWDb1Voo-F9a7ZxJ",
-				Help:  "rclone's client secret - use this or your own if you want",
-			}},
-		}, {
-			Name: "token_file",
-			Help: "Path to store token file.",
-			Examples: []fs.OptionExample{{
-				Value: path.Join(fs.HomeDir, ".gdrive-token-file"),
-				Help:  "Suggested path for token file",
-			}},
 		}},
 	})
 }
 
+// Configuration helper - called after the user has put in the defaults
+func Config(name string) {
+	// See if already have a token
+	tokenString := fs.ConfigFile.MustValue(name, "token")
+	if tokenString != "" {
+		fmt.Printf("Already have a drive token - refresh?\n")
+		if !fs.Confirm() {
+			return
+		}
+	}
+
+	// Get a drive transport
+	t, err := newDriveTransport(name)
+	if err != nil {
+		log.Fatalf("Couldn't make drive transport: %v", err)
+	}
+
+	// Generate a URL for the user to visit for authorization.
+	authUrl := t.Config.AuthCodeURL("state")
+	fmt.Printf("Go to the following link in your browser\n")
+	fmt.Printf("%s\n", authUrl)
+	fmt.Printf("Log in, then type paste the token that is returned in the browser here\n")
+
+	// Read the code, and exchange it for a token.
+	fmt.Printf("Enter verification code> ")
+	authCode := fs.ReadLine()
+	_, err = t.Exchange(authCode)
+	if err != nil {
+		log.Fatalf("Failed to get token: %v", err)
+	}
+
+}
+
+// A token cache to save the token in the config file section named
+type tokenCache string
+
+// Get the token from the config file - returns an error if it isn't present
+func (name tokenCache) Token() (*oauth.Token, error) {
+	tokenString, err := fs.ConfigFile.GetValue(string(name), "token")
+	if err != nil {
+		return nil, err
+	}
+	if tokenString == "" {
+		return nil, fmt.Errorf("Empty token found - please reconfigure")
+	}
+	token := new(oauth.Token)
+	err = json.Unmarshal([]byte(tokenString), token)
+	if err != nil {
+		return nil, err
+	}
+	return token, nil
+
+}
+
+// Save the token to the config file
+//
+// This saves the config file if it changes
+func (name tokenCache) PutToken(token *oauth.Token) error {
+	tokenBytes, err := json.Marshal(token)
+	if err != nil {
+		return err
+	}
+	tokenString := string(tokenBytes)
+	old := fs.ConfigFile.MustValue(string(name), "token")
+	if tokenString != old {
+		fs.ConfigFile.SetValue(string(name), "token", tokenString)
+		fs.SaveConfig()
+	}
+	return nil
+}
+
 // FsDrive represents a remote drive server
 type FsDrive struct {
 	svc         *drive.Service // the connection to the drive server
@@ -141,19 +204,6 @@ func (m *dirCache) Flush() {
 
 // ------------------------------------------------------------
 
-// Constants
-const (
-	//	defaultDriveTokenFile = ".google-drive-token" // FIXME root in home directory somehow
-	driveFolderType = "application/vnd.google-apps.folder"
-)
-
-// Globals
-var (
-	// Flags
-	driveAuthCode = flag.String("drive-auth-code", "", "Pass in when requested to make the drive token file.")
-	driveFullList = flag.Bool("drive-full-list", true, "Use a full listing for directory list. More data but usually quicker.")
-)
-
 // String converts this FsDrive to a string
 func (f *FsDrive) String() string {
 	return fmt.Sprintf("Google drive root '%s'", f.root)
@@ -214,39 +264,15 @@ OUTER:
 	return
 }
 
-// Ask the user for a new auth
+// Makes a new drive transport from the config
-func MakeNewToken(t *oauth.Transport) error {
+func newDriveTransport(name string) (*oauth.Transport, error) {
-	if *driveAuthCode == "" {
-		// Generate a URL to visit for authorization.
-		authUrl := t.Config.AuthCodeURL("state")
-		fmt.Fprintf(os.Stderr, "Go to the following link in your browser\n")
-		fmt.Fprintf(os.Stderr, "%s\n", authUrl)
-		fmt.Fprintf(os.Stderr, "Log in, then re-run this program with the -drive-auth-code parameter\n")
-		fmt.Fprintf(os.Stderr, "You only need this parameter once until the drive token file has been created\n")
-		return errors.New("Re-run with --drive-auth-code")
-	}
-
-	// Read the code, and exchange it for a token.
-	//fmt.Printf("Enter verification code: ")
-	//var code string
-	//fmt.Scanln(&code)
-	_, err := t.Exchange(*driveAuthCode)
-	return err
-}
-
-// NewFs contstructs an FsDrive from the path, container:path
-func NewFs(name, path string) (fs.Fs, error) {
 	clientId := fs.ConfigFile.MustValue(name, "client_id")
 	if clientId == "" {
-		return nil, errors.New("client_id not found")
+		clientId = rcloneClientId
 	}
 	clientSecret := fs.ConfigFile.MustValue(name, "client_secret")
 	if clientSecret == "" {
-		return nil, errors.New("client_secret not found")
+		clientSecret = rcloneClientSecret
-	}
-	tokenFile := fs.ConfigFile.MustValue(name, "token_file")
-	if tokenFile == "" {
-		return nil, errors.New("token-file not found")
 	}
 
 	// Settings for authorization.
@@ -257,7 +283,22 @@ func NewFs(name, path string) (fs.Fs, error) {
 		RedirectURL:  "urn:ietf:wg:oauth:2.0:oob",
 		AuthURL:      "https://accounts.google.com/o/oauth2/auth",
 		TokenURL:     "https://accounts.google.com/o/oauth2/token",
-		TokenCache:   oauth.CacheFile(tokenFile),
+		TokenCache:   tokenCache(name),
+	}
+
+	t := &oauth.Transport{
+		Config:    driveConfig,
+		Transport: http.DefaultTransport,
+	}
+
+	return t, nil
+}
+
+// NewFs contstructs an FsDrive from the path, container:path
+func NewFs(name, path string) (fs.Fs, error) {
+	t, err := newDriveTransport(name)
+	if err != nil {
+		return nil, err
 	}
 
 	root, err := parseDrivePath(path)
@@ -266,22 +307,10 @@ func NewFs(name, path string) (fs.Fs, error) {
 	}
 	f := &FsDrive{root: root, dirCache: newDirCache()}
 
-	t := &oauth.Transport{
-		Config:    driveConfig,
-		Transport: http.DefaultTransport,
-	}
-
 	// Try to pull the token from the cache; if this fails, we need to get one.
-	token, err := driveConfig.TokenCache.Token()
+	token, err := t.Config.TokenCache.Token()
 	if err != nil {
-		err := MakeNewToken(t)
+		return nil, fmt.Errorf("Failed to get token: %s", err)
-		if err != nil {
-			return nil, fmt.Errorf("Failed to authorise: %s", err)
-		}
-	} else {
-		if *driveAuthCode != "" {
-			return nil, fmt.Errorf("Only supply -drive-auth-code once")
-		}
 	}
 	t.Token = token
 

fs/config.go

@@ -25,18 +25,19 @@ const (
 var (
 	// Config file
 	ConfigFile *goconfig.ConfigFile
+	// Home directory
+	HomeDir = configHome()
 	// Config file path
-	ConfigPath string
+	ConfigPath = path.Join(HomeDir, configFileName)
 	// Global config
 	Config = &ConfigInfo{}
-	// Home directory
-	HomeDir string
 	// Flags
 	verbose      = flag.Bool("verbose", false, "Print lots more stuff")
 	quiet        = flag.Bool("quiet", false, "Print as little stuff as possible")
 	modifyWindow = flag.Duration("modify-window", time.Nanosecond, "Max time diff to be considered the same")
 	checkers     = flag.Int("checkers", 8, "Number of checkers to run in parallel.")
 	transfers    = flag.Int("transfers", 4, "Number of file transfers to run in parallel.")
+	configFile   = flag.String("config", ConfigPath, "Config file.")
 )
 
 // Filesystem config options
@@ -48,6 +49,17 @@ type ConfigInfo struct {
 	Transfers    int
 }
 
+// Find the config directory
+func configHome() string {
+	// Find users home directory
+	usr, err := user.Current()
+	if err != nil {
+		log.Printf("Couldn't find home directory: %v", err)
+		return ""
+	}
+	return usr.HomeDir
+}
+
 // Loads the config file
 func LoadConfig() {
 	// Read some flags if set
@@ -59,19 +71,17 @@ func LoadConfig() {
 	Config.Checkers = *checkers
 	Config.Transfers = *transfers
 
-	// Find users home directory
+	ConfigPath = *configFile
-	usr, err := user.Current()
-	if err != nil {
-		log.Printf("Couldn't find home directory: %v", err)
-		return
-	}
-	HomeDir = usr.HomeDir
-	ConfigPath = path.Join(HomeDir, configFileName)
 
 	// Load configuration file.
+	var err error
 	ConfigFile, err = goconfig.LoadConfigFile(ConfigPath)
 	if err != nil {
 		log.Printf("Failed to load config file %v - using defaults", ConfigPath)
+		ConfigFile, err = goconfig.LoadConfigFile(os.DevNull)
+		if err != nil {
+			log.Fatalf("Failed to read null config file: %v", err)
+		}
 	}
 }
 
@@ -81,11 +91,18 @@ func SaveConfig() {
 	if err != nil {
 		log.Fatalf("Failed to save config file: %v", err)
 	}
+	err = os.Chmod(ConfigPath, 0600)
+	if err != nil {
+		log.Printf("Failed to set permissions on config file: %v", err)
+	}
 }
 
 // Show an overview of the config file
-func ShowConfig() {
+func ShowRemotes() {
 	remotes := ConfigFile.GetSectionList()
+	if len(remotes) == 0 {
+		return
+	}
 	sort.Strings(remotes)
 	fmt.Printf("%-20s %s\n", "Name", "Type")
 	fmt.Printf("%-20s %s\n", "====", "====")
@@ -112,7 +129,7 @@ func ReadLine() string {
 }
 
 // Command - choose one
-func Command(commands []string) int {
+func Command(commands []string) byte {
 	opts := []string{}
 	for _, text := range commands {
 		fmt.Printf("%c) %s\n", text[0], text[1:])
@@ -128,11 +145,16 @@ func Command(commands []string) int {
 		}
 		i := strings.IndexByte(optString, result[0])
 		if i >= 0 {
-			return i
+			return result[0]
 		}
 	}
 }
 
+// Asks the user for Yes or No and returns true or false
+func Confirm() bool {
+	return Command([]string{"yYes", "nNo"}) == 'y'
+}
+
 // Choose one of the defaults or type a new string if newOk is set
 func Choose(what string, defaults, help []string, newOk bool) string {
 	fmt.Printf("Choose a number from below")
@@ -179,11 +201,11 @@ func ShowRemote(name string) {
 func OkRemote(name string) bool {
 	ShowRemote(name)
 	switch i := Command([]string{"yYes this is OK", "eEdit this remote", "dDelete this remote"}); i {
-	case 0:
+	case 'y':
 		return true
-	case 1:
+	case 'e':
 		return false
-	case 2:
+	case 'd':
 		ConfigFile.DeleteSection(name)
 		return true
 	default:
@@ -192,6 +214,22 @@ func OkRemote(name string) bool {
 	return false
 }
 
+// Runs the config helper for the remote if needed
+func RemoteConfig(name string) {
+	fmt.Printf("Remote config\n")
+	fsName := ConfigFile.MustValue(name, "type")
+	if fsName == "" {
+		log.Fatalf("Couldn't find type of fs for %q", name)
+	}
+	f, err := Find(fsName)
+	if err != nil {
+		log.Fatalf("Didn't find filing system: %v", err)
+	}
+	if f.Config != nil {
+		f.Config(name)
+	}
+}
+
 // Make a new remote
 func NewRemote(name string) {
 	fmt.Printf("What type of source is it?\n")
@@ -208,6 +246,7 @@ func NewRemote(name string) {
 	for _, option := range fs.Options {
 		ConfigFile.SetValue(name, option.Name, option.Choose())
 	}
+	RemoteConfig(name)
 	if OkRemote(name) {
 		SaveConfig()
 		return
@@ -229,6 +268,7 @@ func EditRemote(name string) {
 				ConfigFile.SetValue(name, key, newValue)
 			}
 		}
+		RemoteConfig(name)
 		if OkRemote(name) {
 			break
 		}
@@ -236,24 +276,37 @@ func EditRemote(name string) {
 	SaveConfig()
 }
 
+// Delete a remote
+func DeleteRemote(name string) {
+	ConfigFile.DeleteSection(name)
+	SaveConfig()
+}
+
 // Edit the config file interactively
 func EditConfig() {
 	for {
-		fmt.Printf("Current remotes:\n\n")
+		haveRemotes := len(ConfigFile.GetSectionList()) != 0
-		ShowConfig()
+		what := []string{"eEdit existing remote", "nNew remote", "dDelete remote", "qQuit config"}
-		fmt.Printf("\n")
+		if haveRemotes {
-		switch i := Command([]string{"eEdit existing remote", "nNew remote", "dDelete remote", "qQuit config"}); i {
+			fmt.Printf("Current remotes:\n\n")
-		case 0:
+			ShowRemotes()
+			fmt.Printf("\n")
+		} else {
+			fmt.Printf("No remotes found - make a new one\n")
+			what = append(what[1:2], what[3])
+		}
+		switch i := Command(what); i {
+		case 'e':
 			name := ChooseRemote()
 			EditRemote(name)
-		case 1:
+		case 'n':
 			fmt.Printf("name> ")
 			name := ReadLine()
 			NewRemote(name)
-		case 2:
+		case 'd':
 			name := ChooseRemote()
-			ConfigFile.DeleteSection(name)
+			DeleteRemote(name)
-		case 3:
+		case 'q':
 			return
 		}
 	}

fs/fs.go

@@ -20,6 +20,7 @@ var (
 type FsInfo struct {
 	Name    string                           // name of this fs
 	NewFs   func(string, string) (Fs, error) // create a new file system
+	Config  func(string)                     // function to call to help with config
 	Options []Option
 }
 

notes.txt

@@ -1,19 +1,10 @@
-Make a config setting
-  * everything can be set on command line, env vars or config file
-  * make a config mode to edit the config
-  * make it so you can make aliases memstore: home: etc
-
-Names
-  * rclone - domain ok, apt ok
-
 Todo
   * Make a test suite which can run on all the given types of fs
   * Copy should use the sync code as it is more efficient at directory listing
-  * Factor fses into own packages
   * FIXME: ls without an argument for buckets/containers?
   * FIXME: More -dry-run checks for object transfer
   * Might be quicker to check md5sums first? for swift <-> swift certainly, and maybe for small files
-  * Ignoring the pseudo directories
+  * swift: Ignoring the pseudo directories
     * if object.PseudoDirectory {
       * fmt.Printf("%9s %19s %s\n", "Directory", "-", fs.Remote())
   * Make Account wrapper
@@ -29,36 +20,17 @@ Todo
   * Add max object size to fs metadata - 5GB for swift, infinite for local, ? for s3
     * tie into -max-size flag
 
-Drive
-  * Do we need the secrets or just the code?  If just the code then
-    can make a web service which does the request on the clients
-    behalf so don't need to expose the client secrets
-  * Apparently we don't need -drive-client-id or -drive-client-secret once we have a token
-  * Make a cgi which we send the user to
-    * It has the client secrets
-    * It gets google to authenticate
-    * It receives the token back
-    * It displays the token to the user to paste in to the code
-    * Should be https really
-  * Sometimes get: Failed to copy: Upload failed: googleapi: Error 403: Rate Limit Exceeded
-    * quota is 100.0 requests/second/user
-
 Ideas
   * could do encryption - put IV into metadata?
   * optimise remote copy container to another container using remote
     copy if local is same as remote - use an optional Copier interface
   * Allow subpaths container:/sub/path
-  * look at auth from env in s3 module - add to swift?
   * support
       * sftp
       * scp
       * Google cloud storage: https://developers.google.com/storage/
-      * Google drive: https://developers.google.com/drive/
       * rsync over ssh
       * dropbox: https://github.com/nickoneill/go-dropbox (no MD5s)
-  * grive seems to have its secrets in the source code which would make things easier!
-
-     
 
 Need to make directory objects otherwise can't upload an empty directory
   * Or could upload empty directories only?
@@ -70,17 +42,6 @@ s3
     * Otherwise can set metadata
   * Returns etag and last modified in bucket list
 
-
 Bugs
 
-local & drive need to delete directories
-
-2013/01/18 16:31:32 Waiting for deletions to finish
-2013/01/18 16:31:32 z3: FIXME Skipping directory
-2013/01/18 16:31:32 z3/x: Deleted
-2013/01/18 16:31:32 Deleting path
-2013/01/18 16:31:32 Rmdir failed: remove z3: directory not empty
-
-------------------------------------------------------------
-
 Non verbose - not sure number transferred got counted up? CHECK