1
0
mirror of https://github.com/rclone/rclone.git synced 2026-01-29 07:43:45 +00:00

Compare commits

..

27 Commits

Author SHA1 Message Date
Nick Craig-Wood
298cda4163 mount test: die after 60 seconds with traceback to debug #3154 2019-06-05 16:50:08 +01:00
Nick Craig-Wood
0408abc20e add -v to tests to debug #3154 2019-06-05 16:50:08 +01:00
Nick Craig-Wood
ac4c8d8dfc cmd/providers: add DefaultStr, ValueStr and Type fields
These fields are auto generated:
- DefaultStr - a string rendering of Default
- ValueStr - a string rendering of Value
- Type - the type of the option
2019-06-05 16:23:42 +01:00
Nick Craig-Wood
e2b6172f7d build: stop using a GITHUB_USER as secure variable to fix build log
Before this change any occurences of `ncw` were escaped as `[secure]`
in the build log which is needless obfuscation and quite confusing.
2019-06-05 16:23:16 +01:00
Nick Craig-Wood
32f2895472 Add garry415 to contributors 2019-06-03 21:12:55 +01:00
garry415
1124c423ee fs: add --ignore-case-sync for forced case insensitivity - fixes #2773 2019-06-03 21:12:10 +01:00
Nick Craig-Wood
cd5a2d80ca Add JorisE to contributors 2019-06-03 21:08:15 +01:00
JorisE
1fe0773da6 docs: Update Microsoft OneDrive numbering
I think an option has been added and now the manual for OneDrive is off by one.
2019-06-03 21:08:00 +01:00
Nick Craig-Wood
5a941cdcdc local: fix preallocate warning on Linux with ZFS
Under Linux, rclone attempts to preallocate files for efficiency.

Before this change, pre-allocation would fail on ZFS with the error

    Failed to pre-allocate: operation not supported

After this change rclone tries a different flag combination for ZFS
then disables pre-allocate if that doesn't work.

Fixes #3066
2019-06-03 18:02:26 +01:00
Nick Craig-Wood
62681e45fb Add Philip Harvey to contributors 2019-06-03 17:34:44 +01:00
Philip Harvey
1a2fb52266 s3: make SetModTime work for GLACIER while syncing - Fixes #3224
Before this change rclone would fail with

    Failed to set modification time: InvalidObjectState: Operation is not valid for the source object's storage class

when attempting to set the modification time of an object in GLACIER.

After this change rclone will re-upload the object as part of a sync if it needs to change the modification time.

See: https://forum.rclone.org/t/suspected-bug-in-s3-or-compatible-sync-logic-to-glacier/10187
2019-06-03 15:28:19 +01:00
Nick Craig-Wood
ec4e7316f2 docs: update advice for avoiding mega logins 2019-05-29 17:37:53 +01:00
buengese
11264c4fb8 jottacloud: update docs regarding devices and mountpoints 2019-05-29 17:16:42 +01:00
buengese
25f7f2b60a jottacloud: Add support for selecting device and mountpoint. fixes #3069 2019-05-29 17:16:42 +01:00
Nick Craig-Wood
e7c20e0bce operations: make move and copy individual files obey --backup-dir
Before this change, when using rclone copy or move with --backup-dir
and the source was a single file, rclone would fail to use the backup
directory.

This change looks up the backup directory in the Fs cache and uses it
as appropriate.

This affects any commands which call operations.MoveFile or
operations.CopyFile which includes rclone move/moveto/copy/copyto
where the source is a single file.

Fixes #3219
2019-05-27 16:14:55 +01:00
Nick Craig-Wood
8ee6034b23 Look for Fs in the cache rather than calling NewFs directly
This will save operations when rclone is used in remote control mode
or with the same remote multiple times in the command line.
2019-05-27 16:14:55 +01:00
Nick Craig-Wood
206e1caa99 fs/cache: factor Fs caching from fs/rc into its own package 2019-05-27 16:14:55 +01:00
Dan Walters
f0e439de0d dlna: improve logging and error handling
Mostly trying to get logging to happen through rclone's log methods.
Added request logging, and a trace parameter that will dump the
entire request/response for debugging when dealing with poorly
written clients.

Also added a flag to specify the device's "Friendly Name" explicitly,
and made an attempt at allowing mime types in addition to video.
2019-05-27 14:42:33 +01:00
Dan Walters
e5464a2a35 dlna: add some additional metadata, headers, and samsung extensions
Again, mostly just copying what I see in other implementations.  This
does seem to have done the trick so that I can now pause, fast forward,
rewind, etc., on my Samsung F series.
2019-05-27 14:42:33 +01:00
Dan Walters
78d38dda56 dlna: icons and compatibility improvements
Brings in icons for devices to display.  Based on what some
other open implementations have done, it's worth having a simple
stub implmentation of ConnectionManagerService.  Advertise
X_MS_MediaReceiverRegistrar as well, which sounds like it
is necessary for certain MSFT devices (like the X-Box.)
2019-05-27 14:42:33 +01:00
Dan Walters
60bb01b22c dlna: refactor the serve mux
Trying to make it a little easier to understand and work on all the
available routes, etc.
2019-05-27 14:42:33 +01:00
Dan Walters
95a74e02c7 dlna: use a template to render the root service descriptor
For various reasons, it seems to make sense to move away from generating
the XML with objects.  Namespace support is minimal in go, the objects we
have are in an upstream project, and some subtitlties seem likely to
cause problems with poorly written clients.

This removes the empty <iconList></iconList>, but is otherwise the
same output.
2019-05-27 14:42:33 +01:00
Dan Walters
d014aef011 dlna: reformat descriptors with tabs
Reduces size of embedded assets.
2019-05-27 14:42:33 +01:00
Dan Walters
be8c23f0b4 dlna: use vfsgen for static assets
As more assets are added, using vfsgen makes things a bit easier.
2019-05-27 14:42:33 +01:00
Nick Craig-Wood
da3b685cd8 vendor: update github.com/pkg/sftp to fix sftp client issues
See: https://forum.rclone.org/t/failed-to-copy-sftp-folder-not-found-c-ftpsites-ssh-fx-failure/9778
See: https://github.com/pkg/sftp/issues/288
2019-05-24 15:03:23 +01:00
Nick Craig-Wood
9aac2d6965 drive: add notes that cleanup works in the background on drive 2019-05-22 20:48:59 +01:00
Gary Kim
81fad0f0e3 docs: fix typo in cache documentation 2019-05-20 19:20:59 +01:00
51 changed files with 1891 additions and 987 deletions

View File

@@ -32,8 +32,9 @@ env:
global:
- GOTAGS=cmount
- GO111MODULE=off
- GITHUB_USER=ncw
- GOTRACEBACK=all
- secure: gU8gCV9R8Kv/Gn0SmCP37edpfIbPoSvsub48GK7qxJdTU628H0KOMiZW/T0gtV5d67XJZ4eKnhJYlxwwxgSgfejO32Rh5GlYEKT/FuVoH0BD72dM1GDFLSrUiUYOdoHvf/BKIFA3dJFT4lk2ASy4Zh7SEoXHG6goBlqUpYx8hVA=
- secure: AMjrMAksDy3QwqGqnvtUg8FL/GNVgNqTqhntLF9HSU0njHhX6YurGGnfKdD9vNHlajPQOewvmBjwNLcDWGn2WObdvmh9Ohep0EmOjZ63kliaRaSSQueSd8y0idfqMQAxep0SObOYbEDVmQh0RCAE9wOVKRaPgw98XvgqWGDq5Tw=
- secure: Uaiveq+/rvQjO03GzvQZV2J6pZfedoFuhdXrLVhhHSeP4ZBca0olw7xaqkabUyP3LkVYXMDSX8EbyeuQT1jfEe5wp5sBdfaDtuYW6heFyjiHIIIbVyBfGXon6db4ETBjOaX/Xt8uktrgNge6qFlj+kpnmpFGxf0jmDLw1zgg7tk=
addons:
apt:

View File

@@ -54,10 +54,10 @@ test: rclone
# Quick test
quicktest:
RCLONE_CONFIG="/notfound" go test $(BUILDTAGS) $(GO_FILES)
RCLONE_CONFIG="/notfound" go test -v $(BUILDTAGS) $(GO_FILES)
racequicktest:
RCLONE_CONFIG="/notfound" go test $(BUILDTAGS) -cpu=2 -race $(GO_FILES)
RCLONE_CONFIG="/notfound" go test -v $(BUILDTAGS) -cpu=2 -race $(GO_FILES)
# Do source code quality checks
check: rclone

View File

@@ -20,6 +20,7 @@ import (
"github.com/ncw/rclone/backend/crypt"
"github.com/ncw/rclone/fs"
"github.com/ncw/rclone/fs/cache"
"github.com/ncw/rclone/fs/config"
"github.com/ncw/rclone/fs/config/configmap"
"github.com/ncw/rclone/fs/config/configstruct"
@@ -481,7 +482,7 @@ func NewFs(name, rootPath string, m configmap.Mapper) (fs.Fs, error) {
return nil, errors.Wrapf(err, "failed to create cache directory %v", f.opt.TempWritePath)
}
f.opt.TempWritePath = filepath.ToSlash(f.opt.TempWritePath)
f.tempFs, err = fs.NewFs(f.opt.TempWritePath)
f.tempFs, err = cache.Get(f.opt.TempWritePath)
if err != nil {
return nil, errors.Wrapf(err, "failed to create temp fs: %v", err)
}

View File

@@ -4,16 +4,40 @@ package local
import (
"os"
"sync/atomic"
"github.com/ncw/rclone/fs"
"golang.org/x/sys/unix"
)
var (
fallocFlags = [...]uint32{
unix.FALLOC_FL_KEEP_SIZE, // Default
unix.FALLOC_FL_KEEP_SIZE | unix.FALLOC_FL_PUNCH_HOLE, // for ZFS #3066
}
fallocFlagsIndex int32
)
// preAllocate the file for performance reasons
func preAllocate(size int64, out *os.File) error {
if size <= 0 {
return nil
}
err := unix.Fallocate(int(out.Fd()), unix.FALLOC_FL_KEEP_SIZE, 0, size)
index := atomic.LoadInt32(&fallocFlagsIndex)
again:
if index >= int32(len(fallocFlags)) {
return nil // Fallocate is disabled
}
flags := fallocFlags[index]
err := unix.Fallocate(int(out.Fd()), flags, 0, size)
if err == unix.ENOTSUP {
// Try the next flags combination
index++
atomic.StoreInt32(&fallocFlagsIndex, index)
fs.Debugf(nil, "preAllocate: got error on fallocate, trying combination %d/%d: %v", index, len(fallocFlags), err)
goto again
}
// FIXME could be doing something here
// if err == unix.ENOSPC {
// log.Printf("No space")

View File

@@ -1736,6 +1736,9 @@ func (o *Object) SetModTime(modTime time.Time) error {
if o.fs.opt.SSEKMSKeyID != "" {
req.SSEKMSKeyId = &o.fs.opt.SSEKMSKeyID
}
if o.fs.opt.StorageClass == "GLACIER" || o.fs.opt.StorageClass == "DEEP_ARCHIVE" {
return fs.ErrorCantSetModTime
}
if o.fs.opt.StorageClass != "" {
req.StorageClass = &o.fs.opt.StorageClass
}

View File

@@ -9,6 +9,7 @@ import (
"time"
"github.com/ncw/rclone/fs"
"github.com/ncw/rclone/fs/cache"
"github.com/ncw/rclone/fs/config/configmap"
"github.com/ncw/rclone/fs/config/configstruct"
"github.com/ncw/rclone/fs/hash"
@@ -342,7 +343,7 @@ func NewFs(name, root string, m configmap.Mapper) (fs.Fs, error) {
if configName != "local" {
rootString = configName + ":" + rootString
}
myFs, err := fs.NewFs(rootString)
myFs, err := cache.Get(rootString)
if err != nil {
if err == fs.ErrorIsFile {
return myFs, err

View File

@@ -22,6 +22,7 @@ import (
"github.com/ncw/rclone/fs"
"github.com/ncw/rclone/fs/accounting"
"github.com/ncw/rclone/fs/cache"
"github.com/ncw/rclone/fs/config/configflags"
"github.com/ncw/rclone/fs/config/flags"
"github.com/ncw/rclone/fs/filter"
@@ -83,7 +84,7 @@ func NewFsFile(remote string) (fs.Fs, string) {
fs.CountError(err)
log.Fatalf("Failed to create file system for %q: %v", remote, err)
}
f, err := fs.NewFs(remote)
f, err := cache.Get(remote)
switch err {
case fs.ErrorIsFile:
return f, path.Base(fsPath)
@@ -131,7 +132,7 @@ func NewFsSrc(args []string) fs.Fs {
//
// This must point to a directory
func newFsDir(remote string) fs.Fs {
f, err := fs.NewFs(remote)
f, err := cache.Get(remote)
if err != nil {
fs.CountError(err)
log.Fatalf("Failed to create file system for %q: %v", remote, err)
@@ -180,7 +181,7 @@ func NewFsSrcDstFiles(args []string) (fsrc fs.Fs, srcFileName string, fdst fs.Fs
log.Fatalf("%q is a directory", args[1])
}
}
fdst, err := fs.NewFs(dstRemote)
fdst, err := cache.Get(dstRemote)
switch err {
case fs.ErrorIsFile:
fs.CountError(err)

View File

@@ -39,6 +39,11 @@ var (
// RunTests runs all the tests against all the VFS cache modes
func RunTests(t *testing.T, fn MountFn) {
// Kill everything if the timer elapes
timer := time.AfterFunc(60*time.Second, func() {
panic("mount has locked up")
})
defer timer.Stop()
mountFn = fn
flag.Parse()
cacheModes := []vfs.CacheMode{

View File

@@ -1,451 +0,0 @@
package dlna
const contentDirectoryServiceDescription = `<?xml version="1.0"?>
<scpd xmlns="urn:schemas-upnp-org:service-1-0">
<specVersion>
<major>1</major>
<minor>0</minor>
</specVersion>
<actionList>
<action>
<name>GetSearchCapabilities</name>
<argumentList>
<argument>
<name>SearchCaps</name>
<direction>out</direction>
<relatedStateVariable>SearchCapabilities</relatedStateVariable>
</argument>
</argumentList>
</action>
<action>
<name>GetSortCapabilities</name>
<argumentList>
<argument>
<name>SortCaps</name>
<direction>out</direction>
<relatedStateVariable>SortCapabilities</relatedStateVariable>
</argument>
</argumentList>
</action>
<action>
<name>GetSortExtensionCapabilities</name>
<argumentList>
<argument>
<name>SortExtensionCaps</name>
<direction>out</direction>
<relatedStateVariable>SortExtensionCapabilities</relatedStateVariable>
</argument>
</argumentList>
</action>
<action>
<name>GetFeatureList</name>
<argumentList>
<argument>
<name>FeatureList</name>
<direction>out</direction>
<relatedStateVariable>FeatureList</relatedStateVariable>
</argument>
</argumentList>
</action>
<action>
<name>GetSystemUpdateID</name>
<argumentList>
<argument>
<name>Id</name>
<direction>out</direction>
<relatedStateVariable>SystemUpdateID</relatedStateVariable>
</argument>
</argumentList>
</action>
<action>
<name>Browse</name>
<argumentList>
<argument>
<name>ObjectID</name>
<direction>in</direction>
<relatedStateVariable>A_ARG_TYPE_ObjectID</relatedStateVariable>
</argument>
<argument>
<name>BrowseFlag</name>
<direction>in</direction>
<relatedStateVariable>A_ARG_TYPE_BrowseFlag</relatedStateVariable>
</argument>
<argument>
<name>Filter</name>
<direction>in</direction>
<relatedStateVariable>A_ARG_TYPE_Filter</relatedStateVariable>
</argument>
<argument>
<name>StartingIndex</name>
<direction>in</direction>
<relatedStateVariable>A_ARG_TYPE_Index</relatedStateVariable>
</argument>
<argument>
<name>RequestedCount</name>
<direction>in</direction>
<relatedStateVariable>A_ARG_TYPE_Count</relatedStateVariable>
</argument>
<argument>
<name>SortCriteria</name>
<direction>in</direction>
<relatedStateVariable>A_ARG_TYPE_SortCriteria</relatedStateVariable>
</argument>
<argument>
<name>Result</name>
<direction>out</direction>
<relatedStateVariable>A_ARG_TYPE_Result</relatedStateVariable>
</argument>
<argument>
<name>NumberReturned</name>
<direction>out</direction>
<relatedStateVariable>A_ARG_TYPE_Count</relatedStateVariable>
</argument>
<argument>
<name>TotalMatches</name>
<direction>out</direction>
<relatedStateVariable>A_ARG_TYPE_Count</relatedStateVariable>
</argument>
<argument>
<name>UpdateID</name>
<direction>out</direction>
<relatedStateVariable>A_ARG_TYPE_UpdateID</relatedStateVariable>
</argument>
</argumentList>
</action>
<action>
<name>Search</name>
<argumentList>
<argument>
<name>ContainerID</name>
<direction>in</direction>
<relatedStateVariable>A_ARG_TYPE_ObjectID</relatedStateVariable>
</argument>
<argument>
<name>SearchCriteria</name>
<direction>in</direction>
<relatedStateVariable>A_ARG_TYPE_SearchCriteria</relatedStateVariable>
</argument>
<argument>
<name>Filter</name>
<direction>in</direction>
<relatedStateVariable>A_ARG_TYPE_Filter</relatedStateVariable>
</argument>
<argument>
<name>StartingIndex</name>
<direction>in</direction>
<relatedStateVariable>A_ARG_TYPE_Index</relatedStateVariable>
</argument>
<argument>
<name>RequestedCount</name>
<direction>in</direction>
<relatedStateVariable>A_ARG_TYPE_Count</relatedStateVariable>
</argument>
<argument>
<name>SortCriteria</name>
<direction>in</direction>
<relatedStateVariable>A_ARG_TYPE_SortCriteria</relatedStateVariable>
</argument>
<argument>
<name>Result</name>
<direction>out</direction>
<relatedStateVariable>A_ARG_TYPE_Result</relatedStateVariable>
</argument>
<argument>
<name>NumberReturned</name>
<direction>out</direction>
<relatedStateVariable>A_ARG_TYPE_Count</relatedStateVariable>
</argument>
<argument>
<name>TotalMatches</name>
<direction>out</direction>
<relatedStateVariable>A_ARG_TYPE_Count</relatedStateVariable>
</argument>
<argument>
<name>UpdateID</name>
<direction>out</direction>
<relatedStateVariable>A_ARG_TYPE_UpdateID</relatedStateVariable>
</argument>
</argumentList>
</action>
<action>
<name>CreateObject</name>
<argumentList>
<argument>
<name>ContainerID</name>
<direction>in</direction>
<relatedStateVariable>A_ARG_TYPE_ObjectID</relatedStateVariable>
</argument>
<argument>
<name>Elements</name>
<direction>in</direction>
<relatedStateVariable>A_ARG_TYPE_Result</relatedStateVariable>
</argument>
<argument>
<name>ObjectID</name>
<direction>out</direction>
<relatedStateVariable>A_ARG_TYPE_ObjectID</relatedStateVariable>
</argument>
<argument>
<name>Result</name>
<direction>out</direction>
<relatedStateVariable>A_ARG_TYPE_Result</relatedStateVariable>
</argument>
</argumentList>
</action>
<action>
<name>DestroyObject</name>
<argumentList>
<argument>
<name>ObjectID</name>
<direction>in</direction>
<relatedStateVariable>A_ARG_TYPE_ObjectID</relatedStateVariable>
</argument>
</argumentList>
</action>
<action>
<name>UpdateObject</name>
<argumentList>
<argument>
<name>ObjectID</name>
<direction>in</direction>
<relatedStateVariable>A_ARG_TYPE_ObjectID</relatedStateVariable>
</argument>
<argument>
<name>CurrentTagValue</name>
<direction>in</direction>
<relatedStateVariable>A_ARG_TYPE_TagValueList</relatedStateVariable>
</argument>
<argument>
<name>NewTagValue</name>
<direction>in</direction>
<relatedStateVariable>A_ARG_TYPE_TagValueList</relatedStateVariable>
</argument>
</argumentList>
</action>
<action>
<name>MoveObject</name>
<argumentList>
<argument>
<name>ObjectID</name>
<direction>in</direction>
<relatedStateVariable>A_ARG_TYPE_ObjectID</relatedStateVariable>
</argument>
<argument>
<name>NewParentID</name>
<direction>in</direction>
<relatedStateVariable>A_ARG_TYPE_ObjectID</relatedStateVariable>
</argument>
<argument>
<name>NewObjectID</name>
<direction>out</direction>
<relatedStateVariable>A_ARG_TYPE_ObjectID</relatedStateVariable>
</argument>
</argumentList>
</action>
<action>
<name>ImportResource</name>
<argumentList>
<argument>
<name>SourceURI</name>
<direction>in</direction>
<relatedStateVariable>A_ARG_TYPE_URI</relatedStateVariable>
</argument>
<argument>
<name>DestinationURI</name>
<direction>in</direction>
<relatedStateVariable>A_ARG_TYPE_URI</relatedStateVariable>
</argument>
<argument>
<name>TransferID</name>
<direction>out</direction>
<relatedStateVariable>A_ARG_TYPE_TransferID</relatedStateVariable>
</argument>
</argumentList>
</action>
<action>
<name>ExportResource</name>
<argumentList>
<argument>
<name>SourceURI</name>
<direction>in</direction>
<relatedStateVariable>A_ARG_TYPE_URI</relatedStateVariable>
</argument>
<argument>
<name>DestinationURI</name>
<direction>in</direction>
<relatedStateVariable>A_ARG_TYPE_URI</relatedStateVariable>
</argument>
<argument>
<name>TransferID</name>
<direction>out</direction>
<relatedStateVariable>A_ARG_TYPE_TransferID</relatedStateVariable>
</argument>
</argumentList>
</action>
<action>
<name>StopTransferResource</name>
<argumentList>
<argument>
<name>TransferID</name>
<direction>in</direction>
<relatedStateVariable>A_ARG_TYPE_TransferID</relatedStateVariable>
</argument>
</argumentList>
</action>
<action>
<name>DeleteResource</name>
<argumentList>
<argument>
<name>ResourceURI</name>
<direction>in</direction>
<relatedStateVariable>A_ARG_TYPE_URI</relatedStateVariable>
</argument>
</argumentList>
</action>
<action>
<name>GetTransferProgress</name>
<argumentList>
<argument>
<name>TransferID</name>
<direction>in</direction>
<relatedStateVariable>A_ARG_TYPE_TransferID</relatedStateVariable>
</argument>
<argument>
<name>TransferStatus</name>
<direction>out</direction>
<relatedStateVariable>A_ARG_TYPE_TransferStatus</relatedStateVariable>
</argument>
<argument>
<name>TransferLength</name>
<direction>out</direction>
<relatedStateVariable>A_ARG_TYPE_TransferLength</relatedStateVariable>
</argument>
<argument>
<name>TransferTotal</name>
<direction>out</direction>
<relatedStateVariable>A_ARG_TYPE_TransferTotal</relatedStateVariable>
</argument>
</argumentList>
</action>
<action>
<name>CreateReference</name>
<argumentList>
<argument>
<name>ContainerID</name>
<direction>in</direction>
<relatedStateVariable>A_ARG_TYPE_ObjectID</relatedStateVariable>
</argument>
<argument>
<name>ObjectID</name>
<direction>in</direction>
<relatedStateVariable>A_ARG_TYPE_ObjectID</relatedStateVariable>
</argument>
<argument>
<name>NewID</name>
<direction>out</direction>
<relatedStateVariable>A_ARG_TYPE_ObjectID</relatedStateVariable>
</argument>
</argumentList>
</action>
</actionList>
<serviceStateTable>
<stateVariable sendEvents="no">
<name>SearchCapabilities</name>
<dataType>string</dataType>
</stateVariable>
<stateVariable sendEvents="no">
<name>SortCapabilities</name>
<dataType>string</dataType>
</stateVariable>
<stateVariable sendEvents="no">
<name>SortExtensionCapabilities</name>
<dataType>string</dataType>
</stateVariable>
<stateVariable sendEvents="yes">
<name>SystemUpdateID</name>
<dataType>ui4</dataType>
</stateVariable>
<stateVariable sendEvents="yes">
<name>ContainerUpdateIDs</name>
<dataType>string</dataType>
</stateVariable>
<stateVariable sendEvents="yes">
<name>TransferIDs</name>
<dataType>string</dataType>
</stateVariable>
<stateVariable sendEvents="no">
<name>FeatureList</name>
<dataType>string</dataType>
</stateVariable>
<stateVariable sendEvents="no">
<name>A_ARG_TYPE_ObjectID</name>
<dataType>string</dataType>
</stateVariable>
<stateVariable sendEvents="no">
<name>A_ARG_TYPE_Result</name>
<dataType>string</dataType>
</stateVariable>
<stateVariable sendEvents="no">
<name>A_ARG_TYPE_SearchCriteria</name>
<dataType>string</dataType>
</stateVariable>
<stateVariable sendEvents="no">
<name>A_ARG_TYPE_BrowseFlag</name>
<dataType>string</dataType>
<allowedValueList>
<allowedValue>BrowseMetadata</allowedValue>
<allowedValue>BrowseDirectChildren</allowedValue>
</allowedValueList>
</stateVariable>
<stateVariable sendEvents="no">
<name>A_ARG_TYPE_Filter</name>
<dataType>string</dataType>
</stateVariable>
<stateVariable sendEvents="no">
<name>A_ARG_TYPE_SortCriteria</name>
<dataType>string</dataType>
</stateVariable>
<stateVariable sendEvents="no">
<name>A_ARG_TYPE_Index</name>
<dataType>ui4</dataType>
</stateVariable>
<stateVariable sendEvents="no">
<name>A_ARG_TYPE_Count</name>
<dataType>ui4</dataType>
</stateVariable>
<stateVariable sendEvents="no">
<name>A_ARG_TYPE_UpdateID</name>
<dataType>ui4</dataType>
</stateVariable>
<stateVariable sendEvents="no">
<name>A_ARG_TYPE_TransferID</name>
<dataType>ui4</dataType>
</stateVariable>
<stateVariable sendEvents="no">
<name>A_ARG_TYPE_TransferStatus</name>
<dataType>string</dataType>
<allowedValueList>
<allowedValue>COMPLETED</allowedValue>
<allowedValue>ERROR</allowedValue>
<allowedValue>IN_PROGRESS</allowedValue>
<allowedValue>STOPPED</allowedValue>
</allowedValueList>
</stateVariable>
<stateVariable sendEvents="no">
<name>A_ARG_TYPE_TransferLength</name>
<dataType>string</dataType>
</stateVariable>
<stateVariable sendEvents="no">
<name>A_ARG_TYPE_TransferTotal</name>
<dataType>string</dataType>
</stateVariable>
<stateVariable sendEvents="no">
<name>A_ARG_TYPE_TagValueList</name>
<dataType>string</dataType>
</stateVariable>
<stateVariable sendEvents="no">
<name>A_ARG_TYPE_URI</name>
<dataType>uri</dataType>
</stateVariable>
</serviceStateTable>
</scpd>`

View File

@@ -9,11 +9,13 @@ import (
"os"
"path"
"path/filepath"
"regexp"
"sort"
"github.com/anacrolix/dms/dlna"
"github.com/anacrolix/dms/upnp"
"github.com/anacrolix/dms/upnpav"
"github.com/ncw/rclone/fs"
"github.com/ncw/rclone/vfs"
"github.com/pkg/errors"
)
@@ -27,6 +29,8 @@ func (cds *contentDirectoryService) updateIDString() string {
return fmt.Sprintf("%d", uint32(os.Getpid()))
}
var mediaMimeTypeRegexp = regexp.MustCompile("^(video|audio|image)/")
// Turns the given entry and DMS host into a UPnP object. A nil object is
// returned if the entry is not of interest.
func (cds *contentDirectoryService) cdsObjectToUpnpavObject(cdsObject object, fileInfo os.FileInfo, host string) (ret interface{}, err error) {
@@ -47,8 +51,13 @@ func (cds *contentDirectoryService) cdsObjectToUpnpavObject(cdsObject object, fi
return
}
// Hardcode "videoItem" so that files show up in VLC.
obj.Class = "object.item.videoItem"
mimeType := fs.MimeTypeFromName(fileInfo.Name())
mediaType := mediaMimeTypeRegexp.FindStringSubmatch(mimeType)
if mediaType == nil {
return
}
obj.Class = "object.item." + mediaType[1] + "Item"
obj.Title = fileInfo.Name()
item := upnpav.Item{
@@ -65,8 +74,7 @@ func (cds *contentDirectoryService) cdsObjectToUpnpavObject(cdsObject object, fi
"path": {cdsObject.Path},
}.Encode(),
}).String(),
// Hardcode "video/x-matroska" so that files show up in VLC.
ProtocolInfo: fmt.Sprintf("http-get:*:video/x-matroska:%s", dlna.ContentFeatures{
ProtocolInfo: fmt.Sprintf("http-get:*:%s:%s", mimeType, dlna.ContentFeatures{
SupportRange: true,
}.String()),
Bitrate: 0,
@@ -106,14 +114,14 @@ func (cds *contentDirectoryService) readContainer(o object, host string) (ret []
}
obj, err := cds.cdsObjectToUpnpavObject(child, de, host)
if err != nil {
log.Printf("error with %s: %s", child.FilePath(), err)
fs.Errorf(cds, "error with %s: %s", child.FilePath(), err)
continue
}
if obj != nil {
ret = append(ret, obj)
} else {
log.Printf("bad %s", de)
if obj == nil {
fs.Debugf(cds, "unrecognized file type: %s", de)
continue
}
ret = append(ret, obj)
}
return
@@ -192,6 +200,14 @@ func (cds *contentDirectoryService) Handle(action string, argsXML []byte, r *htt
"Result": didlLite(string(result)),
"UpdateID": cds.updateIDString(),
}, nil
case "BrowseMetadata":
result, err := xml.Marshal(obj)
if err != nil {
return nil, err
}
return map[string]string{
"Result": didlLite(string(result)),
}, nil
default:
return nil, upnp.Errorf(upnp.ArgumentValueInvalidErrorCode, "unhandled browse flag: %v", browse.BrowseFlag)
}
@@ -199,6 +215,19 @@ func (cds *contentDirectoryService) Handle(action string, argsXML []byte, r *htt
return map[string]string{
"SearchCaps": "",
}, nil
// Samsung Extensions
case "X_GetFeatureList":
return map[string]string{
"FeatureList": `<Features xmlns="urn:schemas-upnp-org:av:avs" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="urn:schemas-upnp-org:av:avs http://www.upnp.org/schemas/av/avs.xsd">
<Feature name="samsung.com_BASICVIEW" version="1">
<container id="/" type="object.item.imageItem"/>
<container id="/" type="object.item.audioItem"/>
<container id="/" type="object.item.videoItem"/>
</Feature>
</Features>`}, nil
case "X_SetBookmark":
// just ignore
return map[string]string{}, nil
default:
return nil, upnp.InvalidActionError
}

View File

@@ -1,184 +0,0 @@
package dlna
const connectionManagerServiceDescription = `<?xml version="1.0" encoding="UTF-8"?>
<scpd xmlns="urn:schemas-upnp-org:service-1-0">
<specVersion>
<major>1</major>
<minor>0</minor>
</specVersion>
<actionList>
<action>
<name>GetProtocolInfo</name>
<argumentList>
<argument>
<name>Source</name>
<direction>out</direction>
<relatedStateVariable>SourceProtocolInfo</relatedStateVariable>
</argument>
<argument>
<name>Sink</name>
<direction>out</direction>
<relatedStateVariable>SinkProtocolInfo</relatedStateVariable>
</argument>
</argumentList>
</action>
<action>
<name>PrepareForConnection</name>
<argumentList>
<argument>
<name>RemoteProtocolInfo</name>
<direction>in</direction>
<relatedStateVariable>A_ARG_TYPE_ProtocolInfo</relatedStateVariable>
</argument>
<argument>
<name>PeerConnectionManager</name>
<direction>in</direction>
<relatedStateVariable>A_ARG_TYPE_ConnectionManager</relatedStateVariable>
</argument>
<argument>
<name>PeerConnectionID</name>
<direction>in</direction>
<relatedStateVariable>A_ARG_TYPE_ConnectionID</relatedStateVariable>
</argument>
<argument>
<name>Direction</name>
<direction>in</direction>
<relatedStateVariable>A_ARG_TYPE_Direction</relatedStateVariable>
</argument>
<argument>
<name>ConnectionID</name>
<direction>out</direction>
<relatedStateVariable>A_ARG_TYPE_ConnectionID</relatedStateVariable>
</argument>
<argument>
<name>AVTransportID</name>
<direction>out</direction>
<relatedStateVariable>A_ARG_TYPE_AVTransportID</relatedStateVariable>
</argument>
<argument>
<name>RcsID</name>
<direction>out</direction>
<relatedStateVariable>A_ARG_TYPE_RcsID</relatedStateVariable>
</argument>
</argumentList>
</action>
<action>
<name>ConnectionComplete</name>
<argumentList>
<argument>
<name>ConnectionID</name>
<direction>in</direction>
<relatedStateVariable>A_ARG_TYPE_ConnectionID</relatedStateVariable>
</argument>
</argumentList>
</action>
<action>
<name>GetCurrentConnectionIDs</name>
<argumentList>
<argument>
<name>ConnectionIDs</name>
<direction>out</direction>
<relatedStateVariable>CurrentConnectionIDs</relatedStateVariable>
</argument>
</argumentList>
</action>
<action>
<name>GetCurrentConnectionInfo</name>
<argumentList>
<argument>
<name>ConnectionID</name>
<direction>in</direction>
<relatedStateVariable>A_ARG_TYPE_ConnectionID</relatedStateVariable>
</argument>
<argument>
<name>RcsID</name>
<direction>out</direction>
<relatedStateVariable>A_ARG_TYPE_RcsID</relatedStateVariable>
</argument>
<argument>
<name>AVTransportID</name>
<direction>out</direction>
<relatedStateVariable>A_ARG_TYPE_AVTransportID</relatedStateVariable>
</argument>
<argument>
<name>ProtocolInfo</name>
<direction>out</direction>
<relatedStateVariable>A_ARG_TYPE_ProtocolInfo</relatedStateVariable>
</argument>
<argument>
<name>PeerConnectionManager</name>
<direction>out</direction>
<relatedStateVariable>A_ARG_TYPE_ConnectionManager</relatedStateVariable>
</argument>
<argument>
<name>PeerConnectionID</name>
<direction>out</direction>
<relatedStateVariable>A_ARG_TYPE_ConnectionID</relatedStateVariable>
</argument>
<argument>
<name>Direction</name>
<direction>out</direction>
<relatedStateVariable>A_ARG_TYPE_Direction</relatedStateVariable>
</argument>
<argument>
<name>Status</name>
<direction>out</direction>
<relatedStateVariable>A_ARG_TYPE_ConnectionStatus</relatedStateVariable>
</argument>
</argumentList>
</action>
</actionList>
<serviceStateTable>
<stateVariable sendEvents="yes">
<name>SourceProtocolInfo</name>
<dataType>string</dataType>
</stateVariable>
<stateVariable sendEvents="yes">
<name>SinkProtocolInfo</name>
<dataType>string</dataType>
</stateVariable>
<stateVariable sendEvents="yes">
<name>CurrentConnectionIDs</name>
<dataType>string</dataType>
</stateVariable>
<stateVariable sendEvents="no">
<name>A_ARG_TYPE_ConnectionStatus</name>
<dataType>string</dataType>
<allowedValueList>
<allowedValue>OK</allowedValue>
<allowedValue>ContentFormatMismatch</allowedValue>
<allowedValue>InsufficientBandwidth</allowedValue>
<allowedValue>UnreliableChannel</allowedValue>
<allowedValue>Unknown</allowedValue>
</allowedValueList>
</stateVariable>
<stateVariable sendEvents="no">
<name>A_ARG_TYPE_ConnectionManager</name>
<dataType>string</dataType>
</stateVariable>
<stateVariable sendEvents="no">
<name>A_ARG_TYPE_Direction</name>
<dataType>string</dataType>
<allowedValueList>
<allowedValue>Input</allowedValue>
<allowedValue>Output</allowedValue>
</allowedValueList>
</stateVariable>
<stateVariable sendEvents="no">
<name>A_ARG_TYPE_ProtocolInfo</name>
<dataType>string</dataType>
</stateVariable>
<stateVariable sendEvents="no">
<name>A_ARG_TYPE_ConnectionID</name>
<dataType>i4</dataType>
</stateVariable>
<stateVariable sendEvents="no">
<name>A_ARG_TYPE_AVTransportID</name>
<dataType>i4</dataType>
</stateVariable>
<stateVariable sendEvents="no">
<name>A_ARG_TYPE_RcsID</name>
<dataType>i4</dataType>
</stateVariable>
</serviceStateTable>
</scpd>`

26
cmd/serve/dlna/cms.go Normal file
View File

@@ -0,0 +1,26 @@
package dlna
import (
"net/http"
"github.com/anacrolix/dms/upnp"
)
const defaultProtocolInfo = "http-get:*:video/mpeg:*,http-get:*:video/mp4:*,http-get:*:video/vnd.dlna.mpeg-tts:*,http-get:*:video/avi:*,http-get:*:video/x-matroska:*,http-get:*:video/x-ms-wmv:*,http-get:*:video/wtv:*,http-get:*:audio/mpeg:*,http-get:*:audio/mp3:*,http-get:*:audio/mp4:*,http-get:*:audio/x-ms-wma*,http-get:*:audio/wav:*,http-get:*:audio/L16:*,http-get:*image/jpeg:*,http-get:*image/png:*,http-get:*image/gif:*,http-get:*image/tiff:*"
type connectionManagerService struct {
*server
upnp.Eventing
}
func (cms *connectionManagerService) Handle(action string, argsXML []byte, r *http.Request) (map[string]string, error) {
switch action {
case "GetProtocolInfo":
return map[string]string{
"Source": defaultProtocolInfo,
"Sink": "",
}, nil
default:
return nil, upnp.InvalidActionError
}
}

View File

@@ -0,0 +1,24 @@
//go:generate go run assets_generate.go
// The "go:generate" directive compiles static assets by running assets_generate.go
// +build ignore
package main
import (
"log"
"net/http"
"github.com/shurcooL/vfsgen"
)
func main() {
var AssetDir http.FileSystem = http.Dir("./static")
err := vfsgen.Generate(AssetDir, vfsgen.Options{
PackageName: "data",
BuildTags: "!dev",
VariableName: "Assets",
})
if err != nil {
log.Fatalln(err)
}
}

File diff suppressed because one or more lines are too long

View File

@@ -0,0 +1,4 @@
//go:generate go run assets_generate.go
// The "go:generate" directive compiles static assets by running assets_generate.go
package data

View File

@@ -0,0 +1,182 @@
<?xml version="1.0" encoding="UTF-8"?>
<scpd xmlns="urn:schemas-upnp-org:service-1-0">
<specVersion>
<major>1</major>
<minor>0</minor>
</specVersion>
<actionList>
<action>
<name>GetProtocolInfo</name>
<argumentList>
<argument>
<name>Source</name>
<direction>out</direction>
<relatedStateVariable>SourceProtocolInfo</relatedStateVariable>
</argument>
<argument>
<name>Sink</name>
<direction>out</direction>
<relatedStateVariable>SinkProtocolInfo</relatedStateVariable>
</argument>
</argumentList>
</action>
<action>
<name>PrepareForConnection</name>
<argumentList>
<argument>
<name>RemoteProtocolInfo</name>
<direction>in</direction>
<relatedStateVariable>A_ARG_TYPE_ProtocolInfo</relatedStateVariable>
</argument>
<argument>
<name>PeerConnectionManager</name>
<direction>in</direction>
<relatedStateVariable>A_ARG_TYPE_ConnectionManager</relatedStateVariable>
</argument>
<argument>
<name>PeerConnectionID</name>
<direction>in</direction>
<relatedStateVariable>A_ARG_TYPE_ConnectionID</relatedStateVariable>
</argument>
<argument>
<name>Direction</name>
<direction>in</direction>
<relatedStateVariable>A_ARG_TYPE_Direction</relatedStateVariable>
</argument>
<argument>
<name>ConnectionID</name>
<direction>out</direction>
<relatedStateVariable>A_ARG_TYPE_ConnectionID</relatedStateVariable>
</argument>
<argument>
<name>AVTransportID</name>
<direction>out</direction>
<relatedStateVariable>A_ARG_TYPE_AVTransportID</relatedStateVariable>
</argument>
<argument>
<name>RcsID</name>
<direction>out</direction>
<relatedStateVariable>A_ARG_TYPE_RcsID</relatedStateVariable>
</argument>
</argumentList>
</action>
<action>
<name>ConnectionComplete</name>
<argumentList>
<argument>
<name>ConnectionID</name>
<direction>in</direction>
<relatedStateVariable>A_ARG_TYPE_ConnectionID</relatedStateVariable>
</argument>
</argumentList>
</action>
<action>
<name>GetCurrentConnectionIDs</name>
<argumentList>
<argument>
<name>ConnectionIDs</name>
<direction>out</direction>
<relatedStateVariable>CurrentConnectionIDs</relatedStateVariable>
</argument>
</argumentList>
</action>
<action>
<name>GetCurrentConnectionInfo</name>
<argumentList>
<argument>
<name>ConnectionID</name>
<direction>in</direction>
<relatedStateVariable>A_ARG_TYPE_ConnectionID</relatedStateVariable>
</argument>
<argument>
<name>RcsID</name>
<direction>out</direction>
<relatedStateVariable>A_ARG_TYPE_RcsID</relatedStateVariable>
</argument>
<argument>
<name>AVTransportID</name>
<direction>out</direction>
<relatedStateVariable>A_ARG_TYPE_AVTransportID</relatedStateVariable>
</argument>
<argument>
<name>ProtocolInfo</name>
<direction>out</direction>
<relatedStateVariable>A_ARG_TYPE_ProtocolInfo</relatedStateVariable>
</argument>
<argument>
<name>PeerConnectionManager</name>
<direction>out</direction>
<relatedStateVariable>A_ARG_TYPE_ConnectionManager</relatedStateVariable>
</argument>
<argument>
<name>PeerConnectionID</name>
<direction>out</direction>
<relatedStateVariable>A_ARG_TYPE_ConnectionID</relatedStateVariable>
</argument>
<argument>
<name>Direction</name>
<direction>out</direction>
<relatedStateVariable>A_ARG_TYPE_Direction</relatedStateVariable>
</argument>
<argument>
<name>Status</name>
<direction>out</direction>
<relatedStateVariable>A_ARG_TYPE_ConnectionStatus</relatedStateVariable>
</argument>
</argumentList>
</action>
</actionList>
<serviceStateTable>
<stateVariable sendEvents="yes">
<name>SourceProtocolInfo</name>
<dataType>string</dataType>
</stateVariable>
<stateVariable sendEvents="yes">
<name>SinkProtocolInfo</name>
<dataType>string</dataType>
</stateVariable>
<stateVariable sendEvents="yes">
<name>CurrentConnectionIDs</name>
<dataType>string</dataType>
</stateVariable>
<stateVariable sendEvents="no">
<name>A_ARG_TYPE_ConnectionStatus</name>
<dataType>string</dataType>
<allowedValueList>
<allowedValue>OK</allowedValue>
<allowedValue>ContentFormatMismatch</allowedValue>
<allowedValue>InsufficientBandwidth</allowedValue>
<allowedValue>UnreliableChannel</allowedValue>
<allowedValue>Unknown</allowedValue>
</allowedValueList>
</stateVariable>
<stateVariable sendEvents="no">
<name>A_ARG_TYPE_ConnectionManager</name>
<dataType>string</dataType>
</stateVariable>
<stateVariable sendEvents="no">
<name>A_ARG_TYPE_Direction</name>
<dataType>string</dataType>
<allowedValueList>
<allowedValue>Input</allowedValue>
<allowedValue>Output</allowedValue>
</allowedValueList>
</stateVariable>
<stateVariable sendEvents="no">
<name>A_ARG_TYPE_ProtocolInfo</name>
<dataType>string</dataType>
</stateVariable>
<stateVariable sendEvents="no">
<name>A_ARG_TYPE_ConnectionID</name>
<dataType>i4</dataType>
</stateVariable>
<stateVariable sendEvents="no">
<name>A_ARG_TYPE_AVTransportID</name>
<dataType>i4</dataType>
</stateVariable>
<stateVariable sendEvents="no">
<name>A_ARG_TYPE_RcsID</name>
<dataType>i4</dataType>
</stateVariable>
</serviceStateTable>
</scpd>

View File

@@ -0,0 +1,504 @@
<?xml version="1.0"?>
<scpd xmlns="urn:schemas-upnp-org:service-1-0">
<specVersion>
<major>1</major>
<minor>0</minor>
</specVersion>
<actionList>
<action>
<name>GetSearchCapabilities</name>
<argumentList>
<argument>
<name>SearchCaps</name>
<direction>out</direction>
<relatedStateVariable>SearchCapabilities</relatedStateVariable>
</argument>
</argumentList>
</action>
<action>
<name>GetSortCapabilities</name>
<argumentList>
<argument>
<name>SortCaps</name>
<direction>out</direction>
<relatedStateVariable>SortCapabilities</relatedStateVariable>
</argument>
</argumentList>
</action>
<action>
<name>GetSortExtensionCapabilities</name>
<argumentList>
<argument>
<name>SortExtensionCaps</name>
<direction>out</direction>
<relatedStateVariable>SortExtensionCapabilities</relatedStateVariable>
</argument>
</argumentList>
</action>
<action>
<name>GetFeatureList</name>
<argumentList>
<argument>
<name>FeatureList</name>
<direction>out</direction>
<relatedStateVariable>FeatureList</relatedStateVariable>
</argument>
</argumentList>
</action>
<action>
<name>GetSystemUpdateID</name>
<argumentList>
<argument>
<name>Id</name>
<direction>out</direction>
<relatedStateVariable>SystemUpdateID</relatedStateVariable>
</argument>
</argumentList>
</action>
<action>
<name>Browse</name>
<argumentList>
<argument>
<name>ObjectID</name>
<direction>in</direction>
<relatedStateVariable>A_ARG_TYPE_ObjectID</relatedStateVariable>
</argument>
<argument>
<name>BrowseFlag</name>
<direction>in</direction>
<relatedStateVariable>A_ARG_TYPE_BrowseFlag</relatedStateVariable>
</argument>
<argument>
<name>Filter</name>
<direction>in</direction>
<relatedStateVariable>A_ARG_TYPE_Filter</relatedStateVariable>
</argument>
<argument>
<name>StartingIndex</name>
<direction>in</direction>
<relatedStateVariable>A_ARG_TYPE_Index</relatedStateVariable>
</argument>
<argument>
<name>RequestedCount</name>
<direction>in</direction>
<relatedStateVariable>A_ARG_TYPE_Count</relatedStateVariable>
</argument>
<argument>
<name>SortCriteria</name>
<direction>in</direction>
<relatedStateVariable>A_ARG_TYPE_SortCriteria</relatedStateVariable>
</argument>
<argument>
<name>Result</name>
<direction>out</direction>
<relatedStateVariable>A_ARG_TYPE_Result</relatedStateVariable>
</argument>
<argument>
<name>NumberReturned</name>
<direction>out</direction>
<relatedStateVariable>A_ARG_TYPE_Count</relatedStateVariable>
</argument>
<argument>
<name>TotalMatches</name>
<direction>out</direction>
<relatedStateVariable>A_ARG_TYPE_Count</relatedStateVariable>
</argument>
<argument>
<name>UpdateID</name>
<direction>out</direction>
<relatedStateVariable>A_ARG_TYPE_UpdateID</relatedStateVariable>
</argument>
</argumentList>
</action>
<action>
<name>Search</name>
<argumentList>
<argument>
<name>ContainerID</name>
<direction>in</direction>
<relatedStateVariable>A_ARG_TYPE_ObjectID</relatedStateVariable>
</argument>
<argument>
<name>SearchCriteria</name>
<direction>in</direction>
<relatedStateVariable>A_ARG_TYPE_SearchCriteria</relatedStateVariable>
</argument>
<argument>
<name>Filter</name>
<direction>in</direction>
<relatedStateVariable>A_ARG_TYPE_Filter</relatedStateVariable>
</argument>
<argument>
<name>StartingIndex</name>
<direction>in</direction>
<relatedStateVariable>A_ARG_TYPE_Index</relatedStateVariable>
</argument>
<argument>
<name>RequestedCount</name>
<direction>in</direction>
<relatedStateVariable>A_ARG_TYPE_Count</relatedStateVariable>
</argument>
<argument>
<name>SortCriteria</name>
<direction>in</direction>
<relatedStateVariable>A_ARG_TYPE_SortCriteria</relatedStateVariable>
</argument>
<argument>
<name>Result</name>
<direction>out</direction>
<relatedStateVariable>A_ARG_TYPE_Result</relatedStateVariable>
</argument>
<argument>
<name>NumberReturned</name>
<direction>out</direction>
<relatedStateVariable>A_ARG_TYPE_Count</relatedStateVariable>
</argument>
<argument>
<name>TotalMatches</name>
<direction>out</direction>
<relatedStateVariable>A_ARG_TYPE_Count</relatedStateVariable>
</argument>
<argument>
<name>UpdateID</name>
<direction>out</direction>
<relatedStateVariable>A_ARG_TYPE_UpdateID</relatedStateVariable>
</argument>
</argumentList>
</action>
<action>
<name>CreateObject</name>
<argumentList>
<argument>
<name>ContainerID</name>
<direction>in</direction>
<relatedStateVariable>A_ARG_TYPE_ObjectID</relatedStateVariable>
</argument>
<argument>
<name>Elements</name>
<direction>in</direction>
<relatedStateVariable>A_ARG_TYPE_Result</relatedStateVariable>
</argument>
<argument>
<name>ObjectID</name>
<direction>out</direction>
<relatedStateVariable>A_ARG_TYPE_ObjectID</relatedStateVariable>
</argument>
<argument>
<name>Result</name>
<direction>out</direction>
<relatedStateVariable>A_ARG_TYPE_Result</relatedStateVariable>
</argument>
</argumentList>
</action>
<action>
<name>DestroyObject</name>
<argumentList>
<argument>
<name>ObjectID</name>
<direction>in</direction>
<relatedStateVariable>A_ARG_TYPE_ObjectID</relatedStateVariable>
</argument>
</argumentList>
</action>
<action>
<name>UpdateObject</name>
<argumentList>
<argument>
<name>ObjectID</name>
<direction>in</direction>
<relatedStateVariable>A_ARG_TYPE_ObjectID</relatedStateVariable>
</argument>
<argument>
<name>CurrentTagValue</name>
<direction>in</direction>
<relatedStateVariable>A_ARG_TYPE_TagValueList</relatedStateVariable>
</argument>
<argument>
<name>NewTagValue</name>
<direction>in</direction>
<relatedStateVariable>A_ARG_TYPE_TagValueList</relatedStateVariable>
</argument>
</argumentList>
</action>
<action>
<name>MoveObject</name>
<argumentList>
<argument>
<name>ObjectID</name>
<direction>in</direction>
<relatedStateVariable>A_ARG_TYPE_ObjectID</relatedStateVariable>
</argument>
<argument>
<name>NewParentID</name>
<direction>in</direction>
<relatedStateVariable>A_ARG_TYPE_ObjectID</relatedStateVariable>
</argument>
<argument>
<name>NewObjectID</name>
<direction>out</direction>
<relatedStateVariable>A_ARG_TYPE_ObjectID</relatedStateVariable>
</argument>
</argumentList>
</action>
<action>
<name>ImportResource</name>
<argumentList>
<argument>
<name>SourceURI</name>
<direction>in</direction>
<relatedStateVariable>A_ARG_TYPE_URI</relatedStateVariable>
</argument>
<argument>
<name>DestinationURI</name>
<direction>in</direction>
<relatedStateVariable>A_ARG_TYPE_URI</relatedStateVariable>
</argument>
<argument>
<name>TransferID</name>
<direction>out</direction>
<relatedStateVariable>A_ARG_TYPE_TransferID</relatedStateVariable>
</argument>
</argumentList>
</action>
<action>
<name>ExportResource</name>
<argumentList>
<argument>
<name>SourceURI</name>
<direction>in</direction>
<relatedStateVariable>A_ARG_TYPE_URI</relatedStateVariable>
</argument>
<argument>
<name>DestinationURI</name>
<direction>in</direction>
<relatedStateVariable>A_ARG_TYPE_URI</relatedStateVariable>
</argument>
<argument>
<name>TransferID</name>
<direction>out</direction>
<relatedStateVariable>A_ARG_TYPE_TransferID</relatedStateVariable>
</argument>
</argumentList>
</action>
<action>
<name>StopTransferResource</name>
<argumentList>
<argument>
<name>TransferID</name>
<direction>in</direction>
<relatedStateVariable>A_ARG_TYPE_TransferID</relatedStateVariable>
</argument>
</argumentList>
</action>
<action>
<name>DeleteResource</name>
<argumentList>
<argument>
<name>ResourceURI</name>
<direction>in</direction>
<relatedStateVariable>A_ARG_TYPE_URI</relatedStateVariable>
</argument>
</argumentList>
</action>
<action>
<name>GetTransferProgress</name>
<argumentList>
<argument>
<name>TransferID</name>
<direction>in</direction>
<relatedStateVariable>A_ARG_TYPE_TransferID</relatedStateVariable>
</argument>
<argument>
<name>TransferStatus</name>
<direction>out</direction>
<relatedStateVariable>A_ARG_TYPE_TransferStatus</relatedStateVariable>
</argument>
<argument>
<name>TransferLength</name>
<direction>out</direction>
<relatedStateVariable>A_ARG_TYPE_TransferLength</relatedStateVariable>
</argument>
<argument>
<name>TransferTotal</name>
<direction>out</direction>
<relatedStateVariable>A_ARG_TYPE_TransferTotal</relatedStateVariable>
</argument>
</argumentList>
</action>
<action>
<name>CreateReference</name>
<argumentList>
<argument>
<name>ContainerID</name>
<direction>in</direction>
<relatedStateVariable>A_ARG_TYPE_ObjectID</relatedStateVariable>
</argument>
<argument>
<name>ObjectID</name>
<direction>in</direction>
<relatedStateVariable>A_ARG_TYPE_ObjectID</relatedStateVariable>
</argument>
<argument>
<name>NewID</name>
<direction>out</direction>
<relatedStateVariable>A_ARG_TYPE_ObjectID</relatedStateVariable>
</argument>
</argumentList>
</action>
<action>
<name>X_GetFeatureList</name>
<argumentList>
<argument>
<name>FeatureList</name>
<direction>out</direction>
<relatedStateVariable>A_ARG_TYPE_Featurelist</relatedStateVariable>
</argument>
</argumentList>
</action>
<action>
<name>X_SetBookmark</name>
<argumentList>
<argument>
<name>CategoryType</name>
<direction>in</direction>
<relatedStateVariable>A_ARG_TYPE_CategoryType</relatedStateVariable>
</argument>
<argument>
<name>RID</name>
<direction>in</direction>
<relatedStateVariable>A_ARG_TYPE_RID</relatedStateVariable>
</argument>
<argument>
<name>ObjectID</name>
<direction>in</direction>
<relatedStateVariable>A_ARG_TYPE_ObjectID</relatedStateVariable>
</argument>
<argument>
<name>PosSecond</name>
<direction>in</direction>
<relatedStateVariable>A_ARG_TYPE_PosSec</relatedStateVariable>
</argument>
</argumentList>
</action>
</actionList>
<serviceStateTable>
<stateVariable sendEvents="no">
<name>SearchCapabilities</name>
<dataType>string</dataType>
</stateVariable>
<stateVariable sendEvents="no">
<name>SortCapabilities</name>
<dataType>string</dataType>
</stateVariable>
<stateVariable sendEvents="no">
<name>SortExtensionCapabilities</name>
<dataType>string</dataType>
</stateVariable>
<stateVariable sendEvents="yes">
<name>SystemUpdateID</name>
<dataType>ui4</dataType>
</stateVariable>
<stateVariable sendEvents="yes">
<name>ContainerUpdateIDs</name>
<dataType>string</dataType>
</stateVariable>
<stateVariable sendEvents="yes">
<name>TransferIDs</name>
<dataType>string</dataType>
</stateVariable>
<stateVariable sendEvents="no">
<name>FeatureList</name>
<dataType>string</dataType>
</stateVariable>
<stateVariable sendEvents="no">
<name>A_ARG_TYPE_ObjectID</name>
<dataType>string</dataType>
</stateVariable>
<stateVariable sendEvents="no">
<name>A_ARG_TYPE_Result</name>
<dataType>string</dataType>
</stateVariable>
<stateVariable sendEvents="no">
<name>A_ARG_TYPE_SearchCriteria</name>
<dataType>string</dataType>
</stateVariable>
<stateVariable sendEvents="no">
<name>A_ARG_TYPE_BrowseFlag</name>
<dataType>string</dataType>
<allowedValueList>
<allowedValue>BrowseMetadata</allowedValue>
<allowedValue>BrowseDirectChildren</allowedValue>
</allowedValueList>
</stateVariable>
<stateVariable sendEvents="no">
<name>A_ARG_TYPE_Filter</name>
<dataType>string</dataType>
</stateVariable>
<stateVariable sendEvents="no">
<name>A_ARG_TYPE_SortCriteria</name>
<dataType>string</dataType>
</stateVariable>
<stateVariable sendEvents="no">
<name>A_ARG_TYPE_Index</name>
<dataType>ui4</dataType>
</stateVariable>
<stateVariable sendEvents="no">
<name>A_ARG_TYPE_Count</name>
<dataType>ui4</dataType>
</stateVariable>
<stateVariable sendEvents="no">
<name>A_ARG_TYPE_UpdateID</name>
<dataType>ui4</dataType>
</stateVariable>
<stateVariable sendEvents="no">
<name>A_ARG_TYPE_TransferID</name>
<dataType>ui4</dataType>
</stateVariable>
<stateVariable sendEvents="no">
<name>A_ARG_TYPE_TransferStatus</name>
<dataType>string</dataType>
<allowedValueList>
<allowedValue>COMPLETED</allowedValue>
<allowedValue>ERROR</allowedValue>
<allowedValue>IN_PROGRESS</allowedValue>
<allowedValue>STOPPED</allowedValue>
</allowedValueList>
</stateVariable>
<stateVariable sendEvents="no">
<name>A_ARG_TYPE_TransferLength</name>
<dataType>string</dataType>
</stateVariable>
<stateVariable sendEvents="no">
<name>A_ARG_TYPE_TransferTotal</name>
<dataType>string</dataType>
</stateVariable>
<stateVariable sendEvents="no">
<name>A_ARG_TYPE_TagValueList</name>
<dataType>string</dataType>
</stateVariable>
<stateVariable sendEvents="no">
<name>A_ARG_TYPE_URI</name>
<dataType>uri</dataType>
</stateVariable>
<stateVariable sendEvents="no">
<name>A_ARG_TYPE_CategoryType</name>
<dataType>ui4</dataType>
<defaultValue />
</stateVariable>
<stateVariable sendEvents="no">
<name>A_ARG_TYPE_RID</name>
<dataType>ui4</dataType>
<defaultValue />
</stateVariable>
<stateVariable sendEvents="no">
<name>A_ARG_TYPE_PosSec</name>
<dataType>ui4</dataType>
<defaultValue />
</stateVariable>
<stateVariable sendEvents="no">
<name>A_ARG_TYPE_Featurelist</name>
<dataType>string</dataType>
<defaultValue />
</stateVariable>
</serviceStateTable>
</scpd>

View File

@@ -0,0 +1,88 @@
<?xml version="1.0" ?>
<scpd xmlns="urn:schemas-upnp-org:service-1-0">
<specVersion>
<major>1</major>
<minor>0</minor>
</specVersion>
<actionList>
<action>
<name>IsAuthorized</name>
<argumentList>
<argument>
<name>DeviceID</name>
<direction>in</direction>
<relatedStateVariable>A_ARG_TYPE_DeviceID</relatedStateVariable>
</argument>
<argument>
<name>Result</name>
<direction>out</direction>
<relatedStateVariable>A_ARG_TYPE_Result</relatedStateVariable>
</argument>
</argumentList>
</action>
<action>
<name>RegisterDevice</name>
<argumentList>
<argument>
<name>RegistrationReqMsg</name>
<direction>in</direction>
<relatedStateVariable>A_ARG_TYPE_RegistrationReqMsg</relatedStateVariable>
</argument>
<argument>
<name>RegistrationRespMsg</name>
<direction>out</direction>
<relatedStateVariable>A_ARG_TYPE_RegistrationRespMsg</relatedStateVariable>
</argument>
</argumentList>
</action>
<action>
<name>IsValidated</name>
<argumentList>
<argument>
<name>DeviceID</name>
<direction>in</direction>
<relatedStateVariable>A_ARG_TYPE_DeviceID</relatedStateVariable>
</argument>
<argument>
<name>Result</name>
<direction>out</direction>
<relatedStateVariable>A_ARG_TYPE_Result</relatedStateVariable>
</argument>
</argumentList>
</action>
</actionList>
<serviceStateTable>
<stateVariable sendEvents="no">
<name>A_ARG_TYPE_DeviceID</name>
<dataType>string</dataType>
</stateVariable>
<stateVariable sendEvents="no">
<name>A_ARG_TYPE_Result</name>
<dataType>int</dataType>
</stateVariable>
<stateVariable sendEvents="no">
<name>A_ARG_TYPE_RegistrationReqMsg</name>
<dataType>bin.base64</dataType>
</stateVariable>
<stateVariable sendEvents="no">
<name>A_ARG_TYPE_RegistrationRespMsg</name>
<dataType>bin.base64</dataType>
</stateVariable>
<stateVariable sendEvents="yes">
<name>AuthorizationGrantedUpdateID</name>
<dataType>ui4</dataType>
</stateVariable>
<stateVariable sendEvents="yes">
<name>AuthorizationDeniedUpdateID</name>
<dataType>ui4</dataType>
</stateVariable>
<stateVariable sendEvents="yes">
<name>ValidationSucceededUpdateID</name>
<dataType>ui4</dataType>
</stateVariable>
<stateVariable sendEvents="yes">
<name>ValidationRevokedUpdateID</name>
<dataType>ui4</dataType>
</stateVariable>
</serviceStateTable>
</scpd>

Binary file not shown.

After

Width:  |  Height:  |  Size: 20 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 5.6 KiB

View File

@@ -4,20 +4,21 @@ import (
"bytes"
"encoding/xml"
"fmt"
"log"
"net"
"net/http"
"net/url"
"os"
"path"
"strconv"
"strings"
"text/template"
"time"
dms_dlna "github.com/anacrolix/dms/dlna"
"github.com/anacrolix/dms/soap"
"github.com/anacrolix/dms/ssdp"
"github.com/anacrolix/dms/upnp"
"github.com/ncw/rclone/cmd"
"github.com/ncw/rclone/cmd/serve/dlna/data"
"github.com/ncw/rclone/cmd/serve/dlna/dlnaflags"
"github.com/ncw/rclone/fs"
"github.com/ncw/rclone/vfs"
@@ -51,7 +52,7 @@ players might show files that they are not able to play back correctly.
cmd.Run(false, false, command, func() error {
s := newServer(f, &dlnaflags.Opt)
if err := s.Serve(); err != nil {
log.Fatal(err)
return err
}
s.Wait()
return nil
@@ -60,60 +61,12 @@ players might show files that they are not able to play back correctly.
}
const (
serverField = "Linux/3.4 DLNADOC/1.50 UPnP/1.0 DMS/1.0"
rootDeviceType = "urn:schemas-upnp-org:device:MediaServer:1"
rootDeviceModelName = "rclone"
resPath = "/res"
rootDescPath = "/rootDesc.xml"
serviceControlURL = "/ctl"
serverField = "Linux/3.4 DLNADOC/1.50 UPnP/1.0 DMS/1.0"
rootDescPath = "/rootDesc.xml"
resPath = "/res"
serviceControlURL = "/ctl"
)
// Groups the service definition with its XML description.
type service struct {
upnp.Service
SCPD string
}
// Exposed UPnP AV services.
var services = []*service{
{
Service: upnp.Service{
ServiceType: "urn:schemas-upnp-org:service:ContentDirectory:1",
ServiceId: "urn:upnp-org:serviceId:ContentDirectory",
ControlURL: serviceControlURL,
},
SCPD: contentDirectoryServiceDescription,
},
{
Service: upnp.Service{
ServiceType: "urn:schemas-upnp-org:service:ConnectionManager:1",
ServiceId: "urn:upnp-org:serviceId:ConnectionManager",
ControlURL: serviceControlURL,
},
SCPD: connectionManagerServiceDescription,
},
}
func init() {
for _, s := range services {
p := path.Join("/scpd", s.ServiceId)
s.SCPDURL = p
}
}
func devices() []string {
return []string{
"urn:schemas-upnp-org:device:MediaServer:1",
}
}
func serviceTypes() (ret []string) {
for _, s := range services {
ret = append(ret, s.ServiceType)
}
return
}
type server struct {
// The service SOAP handler keyed by service URN.
services map[string]UPnPService
@@ -122,10 +75,9 @@ type server struct {
HTTPConn net.Listener
httpListenAddr string
httpServeMux *http.ServeMux
handler http.Handler
rootDeviceUUID string
rootDescXML []byte
RootDeviceUUID string
FriendlyName string
@@ -140,16 +92,16 @@ type server struct {
}
func newServer(f fs.Fs, opt *dlnaflags.Options) *server {
hostName, err := os.Hostname()
if err != nil {
hostName = ""
} else {
hostName = " (" + hostName + ")"
friendlyName := opt.FriendlyName
if friendlyName == "" {
friendlyName = makeDefaultFriendlyName()
}
s := &server{
AnnounceInterval: 10 * time.Second,
FriendlyName: "rclone" + hostName,
FriendlyName: friendlyName,
RootDeviceUUID: makeDeviceUUID(friendlyName),
Interfaces: listInterfaces(),
httpListenAddr: opt.ListenAddr,
@@ -157,35 +109,29 @@ func newServer(f fs.Fs, opt *dlnaflags.Options) *server {
vfs: vfs.New(f, &vfsflags.Opt),
}
s.initServicesMap()
s.listInterfaces()
s.httpServeMux = http.NewServeMux()
s.rootDeviceUUID = makeDeviceUUID(s.FriendlyName)
s.rootDescXML, err = xml.MarshalIndent(
upnp.DeviceDesc{
SpecVersion: upnp.SpecVersion{Major: 1, Minor: 0},
Device: upnp.Device{
DeviceType: rootDeviceType,
FriendlyName: s.FriendlyName,
Manufacturer: "rclone (rclone.org)",
ModelName: rootDeviceModelName,
UDN: s.rootDeviceUUID,
ServiceList: func() (ss []upnp.Service) {
for _, s := range services {
ss = append(ss, s.Service)
}
return
}(),
},
s.services = map[string]UPnPService{
"ContentDirectory": &contentDirectoryService{
server: s,
},
"ConnectionManager": &connectionManagerService{
server: s,
},
" ", " ")
if err != nil {
// Contents are hardcoded, so this will never happen in production.
log.Panicf("Marshal root descriptor XML: %v", err)
}
s.rootDescXML = append([]byte(`<?xml version="1.0"?>`), s.rootDescXML...)
s.initMux(s.httpServeMux)
// Setup the various http routes.
r := http.NewServeMux()
r.HandleFunc(resPath, s.resourceHandler)
if opt.LogTrace {
r.Handle(rootDescPath, traceLogging(http.HandlerFunc(s.rootDescHandler)))
r.Handle(serviceControlURL, traceLogging(http.HandlerFunc(s.serviceControlHandler)))
} else {
r.HandleFunc(rootDescPath, s.rootDescHandler)
r.HandleFunc(serviceControlURL, s.serviceControlHandler)
}
r.Handle("/static/", http.StripPrefix("/static/",
withHeader("Cache-Control", "public, max-age=86400",
http.FileServer(data.Assets))))
s.handler = logging(withHeader("Server", serverField, r))
return s
}
@@ -197,114 +143,137 @@ type UPnPService interface {
Unsubscribe(sid string) error
}
// initServicesMap is called during initialization of the server to prepare some internal datastructures.
func (s *server) initServicesMap() {
urn, err := upnp.ParseServiceType(services[0].ServiceType)
if err != nil {
// The service type is hardcoded, so this error should never happen.
log.Panicf("ParseServiceType: %v", err)
}
s.services = map[string]UPnPService{
urn.Type: &contentDirectoryService{
server: s,
},
}
return
// Formats the server as a string (used for logging.)
func (s *server) String() string {
return fmt.Sprintf("DLNA server on %v", s.httpListenAddr)
}
// listInterfaces is called during initialization of the server to list the network interfaces
// on the machine.
func (s *server) listInterfaces() {
ifs, err := net.Interfaces()
// Returns rclone version number as the model number.
func (s *server) ModelNumber() string {
return fs.Version
}
// Template used to generate the root device XML descriptor.
//
// Due to the use of namespaces and various subtleties with device compatibility,
// it turns out to be easier to use a template than to marshal XML.
//
// For rendering, it is passed the server object for context.
var rootDescTmpl = template.Must(template.New("rootDesc").Parse(`<?xml version="1.0"?>
<root xmlns="urn:schemas-upnp-org:device-1-0"
xmlns:dlna="urn:schemas-dlna-org:device-1-0"
xmlns:sec="http://www.sec.co.kr/dlna">
<specVersion>
<major>1</major>
<minor>0</minor>
</specVersion>
<device>
<deviceType>urn:schemas-upnp-org:device:MediaServer:1</deviceType>
<friendlyName>{{.FriendlyName}}</friendlyName>
<manufacturer>rclone (rclone.org)</manufacturer>
<manufacturerURL>https://rclone.org/</manufacturerURL>
<modelDescription>rclone</modelDescription>
<modelName>rclone</modelName>
<modelNumber>{{.ModelNumber}}</modelNumber>
<modelURL>https://rclone.org/</modelURL>
<serialNumber>00000000</serialNumber>
<UDN>{{.RootDeviceUUID}}</UDN>
<dlna:X_DLNACAP/>
<dlna:X_DLNADOC>DMS-1.50</dlna:X_DLNADOC>
<dlna:X_DLNADOC>M-DMS-1.50</dlna:X_DLNADOC>
<sec:ProductCap>smi,DCM10,getMediaInfo.sec,getCaptionInfo.sec</sec:ProductCap>
<sec:X_ProductCap>smi,DCM10,getMediaInfo.sec,getCaptionInfo.sec</sec:X_ProductCap>
<iconList>
<icon>
<mimetype>image/png</mimetype>
<width>48</width>
<height>48</height>
<depth>8</depth>
<url>/static/rclone-48x48.png</url>
</icon>
<icon>
<mimetype>image/png</mimetype>
<width>120</width>
<height>120</height>
<depth>8</depth>
<url>/static/rclone-120x120.png</url>
</icon>
</iconList>
<serviceList>
<service>
<serviceType>urn:schemas-upnp-org:service:ContentDirectory:1</serviceType>
<serviceId>urn:upnp-org:serviceId:ContentDirectory</serviceId>
<SCPDURL>/static/ContentDirectory.xml</SCPDURL>
<controlURL>/ctl</controlURL>
<eventSubURL></eventSubURL>
</service>
<service>
<serviceType>urn:schemas-upnp-org:service:ConnectionManager:1</serviceType>
<serviceId>urn:upnp-org:serviceId:ConnectionManager</serviceId>
<SCPDURL>/static/ConnectionManager.xml</SCPDURL>
<controlURL>/ctl</controlURL>
<eventSubURL></eventSubURL>
</service>
<service>
<serviceType>urn:microsoft.com:service:X_MS_MediaReceiverRegistrar:1</serviceType>
<serviceId>urn:microsoft.com:serviceId:X_MS_MediaReceiverRegistrar</serviceId>
<SCPDURL>/static/X_MS_MediaReceiverRegistrar.xml</SCPDURL>
<controlURL>/ctl</controlURL>
<eventSubURL></eventSubURL>
</service>
</serviceList>
<presentationURL>/</presentationURL>
</device>
</root>`))
// Renders the root device descriptor.
func (s *server) rootDescHandler(w http.ResponseWriter, r *http.Request) {
buffer := new(bytes.Buffer)
err := rootDescTmpl.Execute(buffer, s)
if err != nil {
fs.Errorf(s.f, "list network interfaces: %v", err)
serveError(s, w, "Failed to create root descriptor XML", err)
return
}
var tmp []net.Interface
for _, intf := range ifs {
if intf.Flags&net.FlagUp == 0 || intf.MTU <= 0 {
continue
}
s.Interfaces = append(s.Interfaces, intf)
tmp = append(tmp, intf)
w.Header().Set("content-type", `text/xml; charset="utf-8"`)
w.Header().Set("cache-control", "private, max-age=60")
w.Header().Set("content-length", strconv.FormatInt(int64(buffer.Len()), 10))
_, err = buffer.WriteTo(w)
if err != nil {
// Network error
fs.Debugf(s, "Error writing rootDesc: %v", err)
}
}
func (s *server) initMux(mux *http.ServeMux) {
mux.HandleFunc(resPath, func(w http.ResponseWriter, r *http.Request) {
remotePath := r.URL.Query().Get("path")
node, err := s.vfs.Stat(remotePath)
if err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
w.Header().Set("Content-Length", strconv.FormatInt(node.Size(), 10))
file := node.(*vfs.File)
in, err := file.Open(os.O_RDONLY)
if err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
}
defer fs.CheckClose(in, &err)
http.ServeContent(w, r, remotePath, node.ModTime(), in)
return
})
mux.HandleFunc(rootDescPath, func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("content-type", `text/xml; charset="utf-8"`)
w.Header().Set("content-length", fmt.Sprint(len(s.rootDescXML)))
w.Header().Set("server", serverField)
_, err := w.Write(s.rootDescXML)
if err != nil {
fs.Errorf(s, "Failed to serve root descriptor XML: %v", err)
}
})
// Install handlers to serve SCPD for each UPnP service.
for _, s := range services {
mux.HandleFunc(s.SCPDURL, func(serviceDesc string) http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("content-type", `text/xml; charset="utf-8"`)
http.ServeContent(w, r, ".xml", time.Time{}, bytes.NewReader([]byte(serviceDesc)))
}
}(s.SCPD))
}
mux.HandleFunc(serviceControlURL, s.serviceControlHandler)
}
// Handle a service control HTTP request.
func (s *server) serviceControlHandler(w http.ResponseWriter, r *http.Request) {
soapActionString := r.Header.Get("SOAPACTION")
soapAction, err := upnp.ParseActionHTTPHeader(soapActionString)
if err != nil {
http.Error(w, err.Error(), http.StatusBadRequest)
serveError(s, w, "Could not parse SOAPACTION header", err)
return
}
var env soap.Envelope
if err := xml.NewDecoder(r.Body).Decode(&env); err != nil {
http.Error(w, err.Error(), http.StatusBadRequest)
serveError(s, w, "Could not parse SOAP request body", err)
return
}
w.Header().Set("Content-Type", `text/xml; charset="utf-8"`)
w.Header().Set("Ext", "")
w.Header().Set("server", serverField)
soapRespXML, code := func() ([]byte, int) {
respArgs, err := s.soapActionResponse(soapAction, env.Body.Action, r)
if err != nil {
fs.Errorf(s, "Error invoking %v: %v", soapAction, err)
upnpErr := upnp.ConvertError(err)
return mustMarshalXML(soap.NewFault("UPnPError", upnpErr)), 500
return mustMarshalXML(soap.NewFault("UPnPError", upnpErr)), http.StatusInternalServerError
}
return marshalSOAPResponse(soapAction, respArgs), 200
return marshalSOAPResponse(soapAction, respArgs), http.StatusOK
}()
bodyStr := fmt.Sprintf(`<?xml version="1.0" encoding="utf-8" standalone="yes"?><s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/" s:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/"><s:Body>%s</s:Body></s:Envelope>`, soapRespXML)
w.WriteHeader(code)
if _, err := w.Write([]byte(bodyStr)); err != nil {
log.Print(err)
fs.Infof(s, "Error writing response: %v", err)
}
}
@@ -318,6 +287,36 @@ func (s *server) soapActionResponse(sa upnp.SoapAction, actionRequestXML []byte,
return service.Handle(sa.Action, actionRequestXML, r)
}
// Serves actual resources (media files).
func (s *server) resourceHandler(w http.ResponseWriter, r *http.Request) {
remotePath := r.URL.Query().Get("path")
node, err := s.vfs.Stat(remotePath)
if err != nil {
http.NotFound(w, r)
return
}
w.Header().Set("Content-Length", strconv.FormatInt(node.Size(), 10))
// add some DLNA specific headers
if r.Header.Get("getContentFeatures.dlna.org") != "" {
w.Header().Set("contentFeatures.dlna.org", dms_dlna.ContentFeatures{
SupportRange: true,
}.String())
}
w.Header().Set("transferMode.dlna.org", "Streaming")
file := node.(*vfs.File)
in, err := file.Open(os.O_RDONLY)
if err != nil {
serveError(node, w, "Could not open resource", err)
return
}
defer fs.CheckClose(in, &err)
http.ServeContent(w, r, remotePath, node.ModTime(), in)
}
// Serve runs the server - returns the error only if
// the listener was not started; does not block, so
// use s.Wait() to block on the listener indefinitely.
@@ -393,13 +392,19 @@ func (s *server) ssdpInterface(intf net.Interface) {
return url.String()
}
// Note that the devices and services advertised here via SSDP should be
// in agreement with the rootDesc XML descriptor that is defined above.
ssdpServer := ssdp.Server{
Interface: intf,
Devices: devices(),
Services: serviceTypes(),
Interface: intf,
Devices: []string{
"urn:schemas-upnp-org:device:MediaServer:1"},
Services: []string{
"urn:schemas-upnp-org:service:ContentDirectory:1",
"urn:schemas-upnp-org:service:ConnectionManager:1",
"urn:microsoft.com:service:X_MS_MediaReceiverRegistrar:1"},
Location: advertiseLocationFn,
Server: serverField,
UUID: s.rootDeviceUUID,
UUID: s.RootDeviceUUID,
NotifyInterval: s.AnnounceInterval,
}
@@ -417,16 +422,16 @@ func (s *server) ssdpInterface(intf net.Interface) {
// good.
return
}
log.Printf("Error creating ssdp server on %s: %s", intf.Name, err)
fs.Errorf(s, "Error creating ssdp server on %s: %s", intf.Name, err)
return
}
defer ssdpServer.Close()
log.Println("Started SSDP on", intf.Name)
fs.Infof(s, "Started SSDP on %v", intf.Name)
stopped := make(chan struct{})
go func() {
defer close(stopped)
if err := ssdpServer.Serve(); err != nil {
log.Printf("%q: %q\n", intf.Name, err)
fs.Errorf(s, "%q: %q\n", intf.Name, err)
}
}()
select {
@@ -438,9 +443,7 @@ func (s *server) ssdpInterface(intf net.Interface) {
func (s *server) serveHTTP() error {
srv := &http.Server{
Handler: http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
s.httpServeMux.ServeHTTP(w, r)
}),
Handler: s.handler,
}
err := srv.Serve(s.HTTPConn)
select {

View File

@@ -6,11 +6,28 @@ import (
"fmt"
"io"
"log"
"net"
"net/http"
"net/http/httptest"
"net/http/httputil"
"os"
"github.com/anacrolix/dms/soap"
"github.com/anacrolix/dms/upnp"
"github.com/ncw/rclone/fs"
)
// Return a default "friendly name" for the server.
func makeDefaultFriendlyName() string {
hostName, err := os.Hostname()
if err != nil {
hostName = ""
} else {
hostName = " (" + hostName + ")"
}
return "rclone" + hostName
}
func makeDeviceUUID(unique string) string {
h := md5.New()
if _, err := io.WriteString(h, unique); err != nil {
@@ -20,6 +37,24 @@ func makeDeviceUUID(unique string) string {
return upnp.FormatUUID(buf)
}
// Get all available active network interfaces.
func listInterfaces() []net.Interface {
ifs, err := net.Interfaces()
if err != nil {
log.Printf("list network interfaces: %v", err)
return []net.Interface{}
}
var active []net.Interface
for _, intf := range ifs {
if intf.Flags&net.FlagUp == 0 || intf.MTU <= 0 {
continue
}
active = append(active, intf)
}
return active
}
func didlLite(chardata string) string {
return `<DIDL-Lite` +
` xmlns:dc="http://purl.org/dc/elements/1.1/"` +
@@ -50,3 +85,99 @@ func marshalSOAPResponse(sa upnp.SoapAction, args map[string]string) []byte {
return []byte(fmt.Sprintf(`<u:%[1]sResponse xmlns:u="%[2]s">%[3]s</u:%[1]sResponse>`,
sa.Action, sa.ServiceURN.String(), mustMarshalXML(soapArgs)))
}
type loggingResponseWriter struct {
http.ResponseWriter
request *http.Request
committed bool
}
func (lrw *loggingResponseWriter) logRequest(code int, err interface{}) {
// Choose appropriate log level based on response status code.
var level fs.LogLevel
if code < 400 && err == nil {
level = fs.LogLevelInfo
} else {
level = fs.LogLevelError
}
fs.LogPrintf(level, lrw.request.URL.Path, "%s %s %d %s %s",
lrw.request.RemoteAddr, lrw.request.Method, code,
lrw.request.Header.Get("SOAPACTION"), err)
}
func (lrw *loggingResponseWriter) WriteHeader(code int) {
lrw.committed = true
lrw.logRequest(code, nil)
lrw.ResponseWriter.WriteHeader(code)
}
// HTTP handler that logs requests and any errors or panics.
func logging(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
lrw := &loggingResponseWriter{ResponseWriter: w, request: r}
defer func() {
err := recover()
if err != nil {
if !lrw.committed {
lrw.logRequest(http.StatusInternalServerError, err)
http.Error(w, fmt.Sprint(err), http.StatusInternalServerError)
} else {
// Too late to send the error to client, but at least log it.
fs.Errorf(r.URL.Path, "Recovered panic: %v", err)
}
}
}()
next.ServeHTTP(lrw, r)
})
}
// HTTP handler that logs complete request and response bodies for debugging.
// Error recovery and general request logging are left to logging().
func traceLogging(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
dump, err := httputil.DumpRequest(r, true)
if err != nil {
serveError(nil, w, "error dumping request", err)
return
}
fs.Debugf(nil, "%s", dump)
recorder := httptest.NewRecorder()
next.ServeHTTP(recorder, r)
dump, err = httputil.DumpResponse(recorder.Result(), true)
if err != nil {
// log the error but ignore it
fs.Errorf(nil, "error dumping response: %v", err)
} else {
fs.Debugf(nil, "%s", dump)
}
// copy from recorder to the real response writer
for k, v := range recorder.Header() {
w.Header()[k] = v
}
w.WriteHeader(recorder.Code)
_, err = recorder.Body.WriteTo(w)
if err != nil {
// Network error
fs.Debugf(nil, "Error writing response: %v", err)
}
})
}
// HTTP handler that sets headers.
func withHeader(name string, value string, next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Header().Set(name, value)
next.ServeHTTP(w, r)
})
}
// serveError returns an http.StatusInternalServerError and logs the error
func serveError(what interface{}, w http.ResponseWriter, text string, err error) {
fs.CountError(err)
fs.Errorf(what, "%s: %v", text, err)
http.Error(w, text+".", http.StatusInternalServerError)
}

View File

@@ -14,16 +14,25 @@ Use --addr to specify which IP address and port the server should
listen on, eg --addr 1.2.3.4:8000 or --addr :8080 to listen to all
IPs.
Use --name to choose the friendly server name, which is by
default "rclone (hostname)".
Use --log-trace in conjunction with -vv to enable additional debug
logging of all UPNP traffic.
`
// Options is the type for DLNA serving options.
type Options struct {
ListenAddr string
ListenAddr string
FriendlyName string
LogTrace bool
}
// DefaultOpt contains the defaults options for DLNA serving.
var DefaultOpt = Options{
ListenAddr: ":7879",
ListenAddr: ":7879",
FriendlyName: "",
LogTrace: false,
}
// Opt contains the options for DLNA serving.
@@ -34,6 +43,8 @@ var (
func addFlagsPrefix(flagSet *pflag.FlagSet, prefix string, Opt *Options) {
rc.AddOption("dlna", &Opt)
flags.StringVarP(flagSet, &Opt.ListenAddr, prefix+"addr", "", Opt.ListenAddr, "ip:port or :port to bind the DLNA http server to.")
flags.StringVarP(flagSet, &Opt.FriendlyName, prefix+"name", "", Opt.FriendlyName, "name of DLNA server")
flags.BoolVarP(flagSet, &Opt.LogTrace, prefix+"log-trace", "", Opt.LogTrace, "enable trace logging of SOAP traffic")
}
// AddFlags add the command line flags for DLNA serving.

View File

@@ -258,3 +258,6 @@ Contributors
* didil <1284255+didil@users.noreply.github.com>
* id01 <gaviniboom@gmail.com>
* Robert Marko <robimarko@gmail.com>
* Philip Harvey <32467456+pharveybattelle@users.noreply.github.com>
* JorisE <JorisE@users.noreply.github.com>
* garry415 <garry.415@gmail.com>

View File

@@ -1,4 +1,4 @@
-----
---
title: "Cache"
description: "Rclone docs for cache remote"
date: "2017-09-03"

View File

@@ -500,6 +500,12 @@ Do a trial run with no permanent changes. Use this to see what rclone
would do without actually doing it. Useful when setting up the `sync`
command which deletes files in the destination.
### --ignore-case-sync ###
Using this option will cause rclone to ignore the case of the files
when synchronizing so files will not be copied/synced when the
existing filenames are the same, even if the casing is different.
### --ignore-checksum ###
Normally rclone will check that the checksums of transferred files

View File

@@ -380,6 +380,10 @@ If you wish to empty your trash you can use the `rclone cleanup remote:`
command which will permanently delete all your trashed files. This command
does not take any path arguments.
Note that Google Drive takes some time (minutes to days) to empty the
trash even though the command returns within a few seconds. No output
is echoed, so there will be no confirmation even using -v or -vv.
### Quota information ###
To view your current quota you can use the `rclone about remote:`

View File

@@ -111,9 +111,10 @@ abnormal situation, for example if you wish to get the public links of
a directory with hundred of files... After more or less a week, the
remote will remote accept rclone logins normally again.
(If you have lots of commands to run one after the other a better way
of doing this might be to run `rclone rcd` and then use `rclone rc` to
run the commands over the API. This will avoid logging in each time.)
You can mitigate this issue by mounting the remote it with `rclone
mount`. This will log-in when mounting and a log-out when unmounting
only. You can also run `rclone rcd` and then use `rclone rc` to run
the commands over the API to avoid logging in each time.
Rclone does not currently close mega sessions (you can see them in the
web interface), however closing the sessions does not solve the issue.

View File

@@ -35,10 +35,10 @@ Type of storage to configure.
Enter a string value. Press Enter for the default ("").
Choose a number from below, or type in your own value
...
17 / Microsoft OneDrive
18 / Microsoft OneDrive
\ "onedrive"
...
Storage> 17
Storage> 18
Microsoft App Client Id
Leave blank normally.
Enter a string value. Press Enter for the default ("").

View File

@@ -268,6 +268,11 @@ files whose local modtime is newer than the time it was last uploaded.
The modified time is stored as metadata on the object as
`X-Amz-Meta-Mtime` as floating point since the epoch accurate to 1 ns.
If the modification time needs to be updated rclone will attempt to perform a server
side copy to update the modification if the object can be copied in a single part.
In the case the object is larger than 5Gb or is in Glacier or Glacier Deep Archive
storage the object will be uploaded rather than copied.
### Multipart uploads ###
rclone supports multipart uploads with S3 which means that it can

92
fs/cache/cache.go vendored Normal file
View File

@@ -0,0 +1,92 @@
// Package cache implements the Fs cache
package cache
import (
"sync"
"time"
"github.com/ncw/rclone/fs"
)
var (
fsCacheMu sync.Mutex
fsCache = map[string]*cacheEntry{}
fsNewFs = fs.NewFs // for tests
expireRunning = false
cacheExpireDuration = 300 * time.Second // expire the cache entry when it is older than this
cacheExpireInterval = 60 * time.Second // interval to run the cache expire
)
type cacheEntry struct {
f fs.Fs // cached f
err error // nil or fs.ErrorIsFile
fsString string // remote string
lastUsed time.Time // time used for expiry
}
// Get gets a fs.Fs named fsString either from the cache or creates it afresh
func Get(fsString string) (f fs.Fs, err error) {
fsCacheMu.Lock()
defer fsCacheMu.Unlock()
entry, ok := fsCache[fsString]
if !ok {
f, err = fsNewFs(fsString)
if err != nil && err != fs.ErrorIsFile {
return f, err
}
entry = &cacheEntry{
f: f,
fsString: fsString,
err: err,
}
fsCache[fsString] = entry
}
entry.lastUsed = time.Now()
if !expireRunning {
time.AfterFunc(cacheExpireInterval, cacheExpire)
expireRunning = true
}
return entry.f, entry.err
}
// Put puts an fs.Fs named fsString into the cache
func Put(fsString string, f fs.Fs) {
fsCacheMu.Lock()
defer fsCacheMu.Unlock()
fsCache[fsString] = &cacheEntry{
f: f,
fsString: fsString,
lastUsed: time.Now(),
}
if !expireRunning {
time.AfterFunc(cacheExpireInterval, cacheExpire)
expireRunning = true
}
}
// cacheExpire expires any entries that haven't been used recently
func cacheExpire() {
fsCacheMu.Lock()
defer fsCacheMu.Unlock()
now := time.Now()
for fsString, entry := range fsCache {
if now.Sub(entry.lastUsed) > cacheExpireDuration {
delete(fsCache, fsString)
}
}
if len(fsCache) != 0 {
time.AfterFunc(cacheExpireInterval, cacheExpire)
expireRunning = true
} else {
expireRunning = false
}
}
// Clear removes everything from the cahce
func Clear() {
fsCacheMu.Lock()
for k := range fsCache {
delete(fsCache, k)
}
fsCacheMu.Unlock()
}

147
fs/cache/cache_test.go vendored Normal file
View File

@@ -0,0 +1,147 @@
package cache
import (
"errors"
"fmt"
"testing"
"time"
"github.com/ncw/rclone/fs"
"github.com/ncw/rclone/fstest/mockfs"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
var (
called = 0
errSentinel = errors.New("an error")
)
func mockNewFs(t *testing.T) func() {
called = 0
oldFsNewFs := fsNewFs
fsNewFs = func(path string) (fs.Fs, error) {
assert.Equal(t, 0, called)
called++
switch path {
case "/":
return mockfs.NewFs("mock", "mock"), nil
case "/file.txt":
return mockfs.NewFs("mock", "mock"), fs.ErrorIsFile
case "/error":
return nil, errSentinel
}
panic(fmt.Sprintf("Unknown path %q", path))
}
return func() {
fsNewFs = oldFsNewFs
fsCacheMu.Lock()
fsCache = map[string]*cacheEntry{}
expireRunning = false
fsCacheMu.Unlock()
}
}
func TestGet(t *testing.T) {
defer mockNewFs(t)()
assert.Equal(t, 0, len(fsCache))
f, err := Get("/")
require.NoError(t, err)
assert.Equal(t, 1, len(fsCache))
f2, err := Get("/")
require.NoError(t, err)
assert.Equal(t, f, f2)
}
func TestGetFile(t *testing.T) {
defer mockNewFs(t)()
assert.Equal(t, 0, len(fsCache))
f, err := Get("/file.txt")
require.Equal(t, fs.ErrorIsFile, err)
assert.Equal(t, 1, len(fsCache))
f2, err := Get("/file.txt")
require.Equal(t, fs.ErrorIsFile, err)
assert.Equal(t, f, f2)
}
func TestGetError(t *testing.T) {
defer mockNewFs(t)()
assert.Equal(t, 0, len(fsCache))
f, err := Get("/error")
require.Equal(t, errSentinel, err)
require.Equal(t, nil, f)
assert.Equal(t, 0, len(fsCache))
}
func TestPut(t *testing.T) {
defer mockNewFs(t)()
f := mockfs.NewFs("mock", "mock")
assert.Equal(t, 0, len(fsCache))
Put("/alien", f)
assert.Equal(t, 1, len(fsCache))
fNew, err := Get("/alien")
require.NoError(t, err)
require.Equal(t, f, fNew)
assert.Equal(t, 1, len(fsCache))
}
func TestCacheExpire(t *testing.T) {
defer mockNewFs(t)()
cacheExpireInterval = time.Millisecond
assert.Equal(t, false, expireRunning)
_, err := Get("/")
require.NoError(t, err)
fsCacheMu.Lock()
entry := fsCache["/"]
assert.Equal(t, 1, len(fsCache))
fsCacheMu.Unlock()
cacheExpire()
fsCacheMu.Lock()
assert.Equal(t, 1, len(fsCache))
entry.lastUsed = time.Now().Add(-cacheExpireDuration - 60*time.Second)
assert.Equal(t, true, expireRunning)
fsCacheMu.Unlock()
time.Sleep(10 * time.Millisecond)
fsCacheMu.Lock()
assert.Equal(t, false, expireRunning)
assert.Equal(t, 0, len(fsCache))
fsCacheMu.Unlock()
}
func TestClear(t *testing.T) {
defer mockNewFs(t)()
assert.Equal(t, 0, len(fsCache))
_, err := Get("/")
require.NoError(t, err)
assert.Equal(t, 1, len(fsCache))
Clear()
assert.Equal(t, 0, len(fsCache))
}

View File

@@ -62,6 +62,7 @@ type ConfigInfo struct {
MaxDepth int
IgnoreSize bool
IgnoreChecksum bool
IgnoreCaseSync bool
NoTraverse bool
NoUpdateModTime bool
DataRateUnit string

View File

@@ -64,6 +64,7 @@ func AddFlags(flagSet *pflag.FlagSet) {
flags.IntVarP(flagSet, &fs.Config.MaxDepth, "max-depth", "", fs.Config.MaxDepth, "If set limits the recursion depth to this.")
flags.BoolVarP(flagSet, &fs.Config.IgnoreSize, "ignore-size", "", false, "Ignore size when skipping use mod-time or checksum.")
flags.BoolVarP(flagSet, &fs.Config.IgnoreChecksum, "ignore-checksum", "", fs.Config.IgnoreChecksum, "Skip post copy check of checksums.")
flags.BoolVarP(flagSet, &fs.Config.IgnoreCaseSync, "ignore-case-sync", "", fs.Config.IgnoreCaseSync, "Ignore case when synchronizing")
flags.BoolVarP(flagSet, &fs.Config.NoTraverse, "no-traverse", "", fs.Config.NoTraverse, "Don't traverse destination file system on copy.")
flags.BoolVarP(flagSet, &fs.Config.NoUpdateModTime, "no-update-modtime", "", fs.Config.NoUpdateModTime, "Don't update destination mod-time if files identical.")
flags.StringVarP(flagSet, &fs.Config.BackupDir, "backup-dir", "", fs.Config.BackupDir, "Make backups into hierarchy based in DIR.")

View File

@@ -2,6 +2,7 @@
package fs
import (
"encoding/json"
"fmt"
"io"
"io/ioutil"
@@ -132,6 +133,29 @@ type Option struct {
Advanced bool // set if this is an advanced config option
}
// BaseOption is an alias for Option used internally
type BaseOption Option
// MarshalJSON turns an Option into JSON
//
// It adds some generated fields for ease of use
// - DefaultStr - a string rendering of Default
// - ValueStr - a string rendering of Value
// - Type - the type of the option
func (o *Option) MarshalJSON() ([]byte, error) {
return json.Marshal(struct {
BaseOption
DefaultStr string
ValueStr string
Type string
}{
BaseOption: BaseOption(*o),
DefaultStr: fmt.Sprint(o.Default),
ValueStr: o.String(),
Type: o.Type(),
})
}
// GetValue gets the current current value which is the default if not set
func (o *Option) GetValue() interface{} {
val := o.Value

View File

@@ -58,7 +58,7 @@ func (m *March) init() {
// | Yes | No | No |
// | No | Yes | Yes |
// | Yes | Yes | Yes |
if m.Fdst.Features().CaseInsensitive {
if m.Fdst.Features().CaseInsensitive || fs.Config.IgnoreCaseSync {
m.transforms = append(m.transforms, strings.ToLower)
}
}

View File

@@ -19,6 +19,7 @@ import (
"github.com/ncw/rclone/fs"
"github.com/ncw/rclone/fs/accounting"
"github.com/ncw/rclone/fs/cache"
"github.com/ncw/rclone/fs/fserrors"
"github.com/ncw/rclone/fs/fshttp"
"github.com/ncw/rclone/fs/hash"
@@ -1475,6 +1476,22 @@ func moveOrCopyFile(fdst fs.Fs, fsrc fs.Fs, dstFileName string, srcFileName stri
}
if NeedTransfer(dstObj, srcObj) {
// If destination already exists, then we must move it into --backup-dir if required
if dstObj != nil && fs.Config.BackupDir != "" {
backupDir, err := cache.Get(fs.Config.BackupDir)
if err != nil {
return errors.Wrap(err, "creating Fs for --backup-dir failed")
}
remoteWithSuffix := SuffixName(dstObj.Remote())
overwritten, _ := backupDir.NewObject(remoteWithSuffix)
_, err = Move(backupDir, overwritten, remoteWithSuffix, dstObj)
if err != nil {
return errors.Wrap(err, "moving to --backup-dir failed")
}
// If successful zero out the dstObj as it is no longer there
dstObj = nil
}
_, err = Op(fdst, dstObj, dstFileName, srcObj)
} else {
accounting.Stats.Checking(srcFileName)

View File

@@ -739,6 +739,29 @@ func TestMoveFile(t *testing.T) {
fstest.CheckItems(t, r.Fremote, file2)
}
func TestMoveFileBackupDir(t *testing.T) {
r := fstest.NewRun(t)
defer r.Finalise()
oldBackupDir := fs.Config.BackupDir
fs.Config.BackupDir = r.FremoteName + "/backup"
defer func() {
fs.Config.BackupDir = oldBackupDir
}()
file1 := r.WriteFile("dst/file1", "file1 contents", t1)
fstest.CheckItems(t, r.Flocal, file1)
file1old := r.WriteObject("dst/file1", "file1 contents old", t1)
fstest.CheckItems(t, r.Fremote, file1old)
err := operations.MoveFile(r.Fremote, r.Flocal, file1.Path, file1.Path)
require.NoError(t, err)
fstest.CheckItems(t, r.Flocal)
file1old.Path = "backup/dst/file1"
fstest.CheckItems(t, r.Fremote, file1old, file1)
}
func TestCopyFile(t *testing.T) {
r := fstest.NewRun(t)
defer r.Finalise()
@@ -765,6 +788,29 @@ func TestCopyFile(t *testing.T) {
fstest.CheckItems(t, r.Fremote, file2)
}
func TestCopyFileBackupDir(t *testing.T) {
r := fstest.NewRun(t)
defer r.Finalise()
oldBackupDir := fs.Config.BackupDir
fs.Config.BackupDir = r.FremoteName + "/backup"
defer func() {
fs.Config.BackupDir = oldBackupDir
}()
file1 := r.WriteFile("dst/file1", "file1 contents", t1)
fstest.CheckItems(t, r.Flocal, file1)
file1old := r.WriteObject("dst/file1", "file1 contents old", t1)
fstest.CheckItems(t, r.Fremote, file1old)
err := operations.CopyFile(r.Fremote, r.Flocal, file1.Path, file1.Path)
require.NoError(t, err)
fstest.CheckItems(t, r.Flocal, file1)
file1old.Path = "backup/dst/file1"
fstest.CheckItems(t, r.Fremote, file1old, file1)
}
// testFsInfo is for unit testing fs.Info
type testFsInfo struct {
name string

View File

@@ -7,6 +7,7 @@ import (
"time"
"github.com/ncw/rclone/fs"
"github.com/ncw/rclone/fs/cache"
"github.com/ncw/rclone/fs/operations"
"github.com/ncw/rclone/fs/rc"
"github.com/ncw/rclone/fstest"
@@ -21,8 +22,8 @@ func rcNewRun(t *testing.T, method string) (*fstest.Run, *rc.Call) {
r := fstest.NewRun(t)
call := rc.Calls.Get(method)
assert.NotNil(t, call)
rc.PutCachedFs(r.LocalName, r.Flocal)
rc.PutCachedFs(r.FremoteName, r.Fremote)
cache.Put(r.LocalName, r.Flocal)
cache.Put(r.FremoteName, r.Fremote)
return r, call
}

View File

@@ -1,86 +1,12 @@
// This implements the Fs cache
// Utilities for accessing the Fs cache
package rc
import (
"sync"
"time"
"github.com/ncw/rclone/fs"
"github.com/ncw/rclone/fs/cache"
)
var (
fsCacheMu sync.Mutex
fsCache = map[string]*cacheEntry{}
fsNewFs = fs.NewFs // for tests
expireRunning = false
cacheExpireDuration = 300 * time.Second // expire the cache entry when it is older than this
cacheExpireInterval = 60 * time.Second // interval to run the cache expire
)
type cacheEntry struct {
f fs.Fs
fsString string
lastUsed time.Time
}
// GetCachedFs gets a fs.Fs named fsString either from the cache or creates it afresh
func GetCachedFs(fsString string) (f fs.Fs, err error) {
fsCacheMu.Lock()
defer fsCacheMu.Unlock()
entry, ok := fsCache[fsString]
if !ok {
f, err = fsNewFs(fsString)
if err != nil {
return nil, err
}
entry = &cacheEntry{
f: f,
fsString: fsString,
}
fsCache[fsString] = entry
}
entry.lastUsed = time.Now()
if !expireRunning {
time.AfterFunc(cacheExpireInterval, cacheExpire)
expireRunning = true
}
return entry.f, err
}
// PutCachedFs puts an fs.Fs named fsString into the cache
func PutCachedFs(fsString string, f fs.Fs) {
fsCacheMu.Lock()
defer fsCacheMu.Unlock()
fsCache[fsString] = &cacheEntry{
f: f,
fsString: fsString,
lastUsed: time.Now(),
}
if !expireRunning {
time.AfterFunc(cacheExpireInterval, cacheExpire)
expireRunning = true
}
}
// cacheExpire expires any entries that haven't been used recently
func cacheExpire() {
fsCacheMu.Lock()
defer fsCacheMu.Unlock()
now := time.Now()
for fsString, entry := range fsCache {
if now.Sub(entry.lastUsed) > cacheExpireDuration {
delete(fsCache, fsString)
}
}
if len(fsCache) != 0 {
time.AfterFunc(cacheExpireInterval, cacheExpire)
expireRunning = true
} else {
expireRunning = false
}
}
// GetFsNamed gets a fs.Fs named fsName either from the cache or creates it afresh
func GetFsNamed(in Params, fsName string) (f fs.Fs, err error) {
fsString, err := in.GetString(fsName)
@@ -88,7 +14,7 @@ func GetFsNamed(in Params, fsName string) (f fs.Fs, err error) {
return nil, err
}
return GetCachedFs(fsString)
return cache.Get(fsString)
}
// GetFs gets a fs.Fs named "fs" either from the cache or creates it afresh

View File

@@ -2,77 +2,21 @@ package rc
import (
"testing"
"time"
"github.com/ncw/rclone/fs"
"github.com/ncw/rclone/fs/cache"
"github.com/ncw/rclone/fstest/mockfs"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
var called = 0
func mockNewFs(t *testing.T) func() {
called = 0
oldFsNewFs := fsNewFs
fsNewFs = func(path string) (fs.Fs, error) {
assert.Equal(t, 0, called)
called++
assert.Equal(t, "/", path)
return mockfs.NewFs("mock", "mock"), nil
}
f := mockfs.NewFs("mock", "mock")
cache.Put("/", f)
return func() {
fsNewFs = oldFsNewFs
fsCacheMu.Lock()
fsCache = map[string]*cacheEntry{}
expireRunning = false
fsCacheMu.Unlock()
cache.Clear()
}
}
func TestGetCachedFs(t *testing.T) {
defer mockNewFs(t)()
assert.Equal(t, 0, len(fsCache))
f, err := GetCachedFs("/")
require.NoError(t, err)
assert.Equal(t, 1, len(fsCache))
f2, err := GetCachedFs("/")
require.NoError(t, err)
assert.Equal(t, f, f2)
}
func TestCacheExpire(t *testing.T) {
defer mockNewFs(t)()
cacheExpireInterval = time.Millisecond
assert.Equal(t, false, expireRunning)
_, err := GetCachedFs("/")
require.NoError(t, err)
fsCacheMu.Lock()
entry := fsCache["/"]
assert.Equal(t, 1, len(fsCache))
fsCacheMu.Unlock()
cacheExpire()
fsCacheMu.Lock()
assert.Equal(t, 1, len(fsCache))
entry.lastUsed = time.Now().Add(-cacheExpireDuration - 60*time.Second)
assert.Equal(t, true, expireRunning)
fsCacheMu.Unlock()
time.Sleep(10 * time.Millisecond)
fsCacheMu.Lock()
assert.Equal(t, false, expireRunning)
assert.Equal(t, 0, len(fsCache))
fsCacheMu.Unlock()
}
func TestGetFsNamed(t *testing.T) {
defer mockNewFs(t)()

View File

@@ -13,6 +13,7 @@ import (
"github.com/ncw/rclone/cmd/serve/httplib"
"github.com/ncw/rclone/cmd/serve/httplib/serve"
"github.com/ncw/rclone/fs"
"github.com/ncw/rclone/fs/cache"
"github.com/ncw/rclone/fs/config"
"github.com/ncw/rclone/fs/list"
"github.com/ncw/rclone/fs/rc"
@@ -222,7 +223,7 @@ func (s *Server) serveRoot(w http.ResponseWriter, r *http.Request) {
}
func (s *Server) serveRemote(w http.ResponseWriter, r *http.Request, path string, fsName string) {
f, err := rc.GetCachedFs(fsName)
f, err := cache.Get(fsName)
if err != nil {
writeError(path, nil, w, errors.Wrap(err, "failed to make Fs"), http.StatusInternalServerError)
return

View File

@@ -3,6 +3,7 @@ package sync
import (
"testing"
"github.com/ncw/rclone/fs/cache"
"github.com/ncw/rclone/fs/rc"
"github.com/ncw/rclone/fstest"
"github.com/stretchr/testify/assert"
@@ -16,8 +17,8 @@ func rcNewRun(t *testing.T, method string) (*fstest.Run, *rc.Call) {
r := fstest.NewRun(t)
call := rc.Calls.Get(method)
assert.NotNil(t, call)
rc.PutCachedFs(r.LocalName, r.Flocal)
rc.PutCachedFs(r.FremoteName, r.Fremote)
cache.Put(r.LocalName, r.Flocal)
cache.Put(r.FremoteName, r.Fremote)
return r, call
}

View File

@@ -10,6 +10,7 @@ import (
"github.com/ncw/rclone/fs"
"github.com/ncw/rclone/fs/accounting"
"github.com/ncw/rclone/fs/cache"
"github.com/ncw/rclone/fs/filter"
"github.com/ncw/rclone/fs/fserrors"
"github.com/ncw/rclone/fs/hash"
@@ -123,7 +124,7 @@ func newSyncCopyMove(fdst, fsrc fs.Fs, deleteMode fs.DeleteMode, DoMove bool, de
// Make Fs for --backup-dir if required
if fs.Config.BackupDir != "" {
var err error
s.backupDir, err = fs.NewFs(fs.Config.BackupDir)
s.backupDir, err = cache.Get(fs.Config.BackupDir)
if err != nil {
return nil, fserrors.FatalError(errors.Errorf("Failed to make fs for --backup-dir %q: %v", fs.Config.BackupDir, err))
}

View File

@@ -1346,6 +1346,33 @@ func TestSyncImmutable(t *testing.T) {
fstest.CheckItems(t, r.Fremote, file1)
}
// Test --ignore-case-sync
func TestSyncIgnoreCase(t *testing.T) {
r := fstest.NewRun(t)
defer r.Finalise()
// Only test if filesystems are case sensitive
if r.Fremote.Features().CaseInsensitive || r.Flocal.Features().CaseInsensitive {
t.Skip("Skipping test as local or remote are case-insensitive")
}
fs.Config.IgnoreCaseSync = true
defer func() { fs.Config.IgnoreCaseSync = false }()
// Create files with different filename casing
file1 := r.WriteFile("existing", "potato", t1)
fstest.CheckItems(t, r.Flocal, file1)
file2 := r.WriteObject("EXISTING", "potato", t1)
fstest.CheckItems(t, r.Fremote, file2)
// Should not copy files that are differently-cased but otherwise identical
accounting.Stats.ResetCounters()
err := Sync(r.Fremote, r.Flocal, false)
require.NoError(t, err)
fstest.CheckItems(t, r.Flocal, file1)
fstest.CheckItems(t, r.Fremote, file2)
}
// Test that aborting on max upload works
func TestAbort(t *testing.T) {
r := fstest.NewRun(t)

2
go.mod
View File

@@ -38,7 +38,7 @@ require (
github.com/patrickmn/go-cache v2.1.0+incompatible
github.com/pengsrc/go-shared v0.2.0 // indirect
github.com/pkg/errors v0.8.1
github.com/pkg/sftp v1.10.0
github.com/pkg/sftp v1.10.1-0.20190523025818-e98a7bef6829
github.com/rfjakob/eme v0.0.0-20171028163933-2222dbd4ba46
github.com/sevlyar/go-daemon v0.1.4
github.com/skratchdot/open-golang v0.0.0-20190402232053-79abb63cd66e

2
go.sum
View File

@@ -156,6 +156,8 @@ github.com/pkg/errors v0.8.1 h1:iURUrRGxPUNPdy5/HRSm+Yj6okJ6UtLINN0Q9M4+h3I=
github.com/pkg/errors v0.8.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
github.com/pkg/sftp v1.10.0 h1:DGA1KlA9esU6WcicH+P8PxFZOl15O6GYtab1cIJdOlE=
github.com/pkg/sftp v1.10.0/go.mod h1:NxmoDg/QLVWluQDUYG7XBZTLUpKeFa8e3aMf1BfjyHk=
github.com/pkg/sftp v1.10.1-0.20190523025818-e98a7bef6829 h1:I+1BDgqX1nXLUL5Uio219Nj0Tne+xW0gp2EOENDk00M=
github.com/pkg/sftp v1.10.1-0.20190523025818-e98a7bef6829/go.mod h1:NxmoDg/QLVWluQDUYG7XBZTLUpKeFa8e3aMf1BfjyHk=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/prometheus/client_golang v0.9.1/go.mod h1:7SWBe2y4D6OKWSNQJUaRYU/AaXPKyh/dDVn+NZz0KFw=

View File

@@ -4,8 +4,8 @@ go_import_path: github.com/pkg/sftp
# current and previous stable releases, plus tip
# remember to exclude previous and tip for macs below
go:
- 1.10.x
- 1.11.x
- 1.12.x
- tip
os:

View File

@@ -884,7 +884,7 @@ func (f *File) Read(b []byte) (int, error) {
// maximise throughput for transferring the entire file (especially
// over high latency links).
func (f *File) WriteTo(w io.Writer) (int64, error) {
fi, err := f.Stat()
fi, err := f.c.Stat(f.path)
if err != nil {
return 0, err
}

2
vendor/modules.txt vendored
View File

@@ -132,7 +132,7 @@ github.com/pengsrc/go-shared/check
github.com/pengsrc/go-shared/reopen
# github.com/pkg/errors v0.8.1
github.com/pkg/errors
# github.com/pkg/sftp v1.10.0
# github.com/pkg/sftp v1.10.1-0.20190523025818-e98a7bef6829
github.com/pkg/sftp
# github.com/pmezard/go-difflib v1.0.0
github.com/pmezard/go-difflib/difflib

View File

@@ -16,6 +16,7 @@ import (
"github.com/djherbis/times"
"github.com/ncw/rclone/fs"
fscache "github.com/ncw/rclone/fs/cache"
"github.com/ncw/rclone/fs/config"
"github.com/pkg/errors"
)
@@ -100,7 +101,7 @@ func newCache(ctx context.Context, f fs.Fs, opt *Options) (*cache, error) {
root := filepath.Join(config.CacheDir, "vfs", f.Name(), fRoot)
fs.Debugf(nil, "vfs cache root is %q", root)
f, err := fs.NewFs(root)
f, err := fscache.Get(root)
if err != nil {
return nil, errors.Wrap(err, "failed to create cache remote")
}