Add support for S3 Object Lock with the following new options:
- --s3-object-lock-mode: set retention mode (GOVERNANCE/COMPLIANCE/copy)
- --s3-object-lock-retain-until-date: set retention date (RFC3339/duration/copy)
- --s3-object-lock-legal-hold-status: set legal hold (ON/OFF/copy)
- --s3-bypass-governance-retention: bypass GOVERNANCE lock on delete
- --s3-bucket-object-lock-enabled: enable Object Lock on bucket creation
- --s3-object-lock-set-after-upload: apply lock via separate API calls
The special value "copy" preserves the source object's setting when used
with --metadata flag, enabling scenarios like cloning objects from
COMPLIANCE to GOVERNANCE mode while preserving the original retention date.
Includes integration tests that create a temporary Object Lock bucket covering:
- Retention Mode and Date
- Legal Hold
- Apply settings after upload
- Override protections using bypass-governance flag
The tests are gracefully skipped on providers that do not support Object Lock.
Fixes#4683Closes#7894#7893#8866
Use URLPathEscapeAll instead of URLPathEscape for path encoding.
URLPathEscape relies on Go's url.URL.String() which only minimally
escapes paths - reserved sub-delimiter characters like semicolons and
equals signs pass through unescaped. Per RFC 3986 section 3.3, these
characters must be percent-encoded when used as literal values in
path segments.
Some WebDAV servers (notably dCache/Jetty) interpret unescaped
semicolons as path parameter delimiters, which truncates filenames
at the semicolon position. URLPathEscapeAll encodes everything
except [A-Za-z0-9/], which is safe for all servers.
Fixes#9082
StackPath's object storage service no longer exists and all S3
endpoints are no longer operational.
Before this change, users could select StackPath as an S3 provider
during configuration, but connections would fail as the endpoints no
longer respond and the service has been discontinued.
After this change, StackPath is removed from the list of supported
S3 providers, preventing users from attempting to configure a
non-functional service.
Fixes#9148
- Add new Fastly provider with US East, US West, and EU Central regions
- Add `etag_is_not_md5` quirk for providers with mandatory encryption
- Disable server-side copy for Fastly (not supported)
Before this change, we uses an unguarded type assertion which failed
when NewObject returned a google document instead of an Object.
This change uses the ID() method to read the id which works for all
types of file instead and returns a sensible error if one isn't found.
Fixes#8990
Before this change it was giving back "401 Unauthorized" however
removing the "Authorization: XXXX" from the request fixes the problem
as the auth is in the URL already.
This updates rclone to use forks of the upstream proton drive modules
in preparation for making changes.
The go-proton-api modules has had changes from master merged so rclone
and Proton-API-Bridge are using the same version.
This was being done in NewFs instead of NewFsWithConnection like it
should have been which meant calls to NewFsWithConnection were not
initialising the proxy correctly.
Fixes previous pull request #8978
An oversight meant that unrestricted API keys
never called b2_list_buckets,
meaning the root remote could not be listed.
The call is now made in the event there are no allowed buckets,
indicating an unrestricted API key
Fixes#9007
When specifying a custom endpoint with a subpath, there is a limitation
in the Google cloud storage integration that the subpath is ignored
during upload operations. For example with the custom endpoint
"example.org/custom/endpoint" on upload the /custom/endpoint is not
reflected.
As this is most likely an issue with the underlying API client, there is
no way to fix this in rclone. By extending the documentation at least
rclone users are made aware of this limitation.
Related forum thread: https://forum.rclone.org/t/googlecloudstorage-custom-endpoint-subpath-removed-for-upload/53059
