Move files to and from a Computer

A Computer’s saved files can be uploaded and downloaded at any time, whether or not the Computer is running. Downloads read saved files and never wake the Computer.

Upload files

Copy a file or a folder into the Computer:

idyl computer cp ./report.txt <computer>:report.txt
idyl computer cp ./assets <computer>:assets

Output similar to:

✓ saved as v4

Each upload publishes one saved version, however many files it holds. A folder’s contents are placed under the destination path, and an empty path places them at the Computer’s root. File modes and symbolic links are kept; ownership and timestamps are not. An upload that would go past a storage limit is refused, publishes no version, and names the limit it would pass. To upload a POSIX tar archive from standard input, use - as the local path:

tar -cf - ./assets | idyl computer cp - <computer>:assets

Continue an interrupted upload

If an upload is interrupted, the CLI prints the key that identifies it:

→ Operation <operation> may still complete; retry the same request with --idempotency-key <key>.

Run the same command again with that key. The upload continues from the bytes already received and publishes one version:

idyl computer cp ./assets <computer>:assets --idempotency-key <key>

Download files

Copy a saved file or folder to this machine, or download every saved file of the Computer into a directory:

idyl computer cp <computer>:report.txt ./report.txt
idyl computer cp <computer>:assets ./assets
idyl computer download <computer> ./backup

A download reads the latest saved version unless you name one, as in <computer>@v2:assets. Changes a running Computer has not saved are not included; add --save-first to cp to save the current files first and read exactly that version. Every file is checked against its saved content hash before it takes its name. Until then, a file waits under a hidden .idyl- name beside its destination, and a folder’s files wait in .idyl-download at the top of the directory, which a finished download removes. Permission bits are kept; ownership and timestamps are not. To write a saved folder to standard output as a POSIX tar archive, use -:

idyl computer cp <computer>:assets - | tar -tf -

Continue an interrupted download

A connection lost part way through is picked up where it stopped. If a download is stopped, run the same command again:

→ Run the same command to continue.

The next run fetches only what is missing: a file already complete is checked and kept, and a file partly fetched continues from what it holds. With --output json, the result states what the run read and what it kept:

idyl computer download <computer> ./backup --output json

Output similar to:

{
  "version": 2,
  "size": 283546112,
  "received": 170189378,
  "kept": 113258553,
  "files": 203,
  "versionMoved": false
}

Field	Meaning
`version`	The saved version read.
`size`	The length of the content read, in bytes.
`received`	The bytes this run fetched.
`kept`	The bytes found complete on this machine from an earlier run.
`files`	The files in place.
`versionMoved`	Whether the latest version changed during the run, in which case the download continued from the version that was latest by then.

An Account with a download limit is told when it can continue. Reaching the limit stops the download with exit code 1, and Ctrl-C stops it with exit code 130. What was fetched is kept either way. A download link reads one saved file or folder at one version. Anyone who holds the link can read those bytes until it expires, with no other credential, so handle it as a secret. A link never saves the Computer first. In the console, Download hands a link to the browser, whose download manager shows progress and resumes after a break. With the TypeScript SDK:

const link = await computer.files.downloadLink("data.bin");
console.log(link.url, link.expiresAt, link.version);

With the Go SDK:

path := "data.bin"
link, err := client.Account().Computers("default").CreateFileDownloadLink(ctx, "my-computer",
	api.CreateFileDownloadLinkRequest{FilePath: api.FilePath{Path: &path}})

A link answers GET and HEAD, and supports a single Range request with If-Range, so a download manager or a program such as curl -C - can resume it.

Download from code

The SDKs download with the same checks and resuming as the CLI. TypeScript:

const file = await computer.files.download("data.bin", {
  onProgress: ({ done, total }) => console.log(done, "of", total),
});
console.log(file.size, file.etag, file.version);
for await (const chunk of file.stream()) {
  // each chunk has been checked against the file's content hash
}

const src = await computer.files.download("src", { archive: true }); // a tar stream

A TypeScript download resumes within the call after a break in the connection. Go:

path := "data.bin"
result, err := client.Account().Computers("default").DownloadFile(ctx, "my-computer",
	api.FilePath{Path: &path}, "./data.bin", idyl.DownloadOptions{})

DownloadFile writes the file under its name only once every byte is checked, and a later call with the same destination fetches only what is missing. DownloadTree downloads a folder in one stream and places each entry through a TreePlacer you supply, which receives each file once its bytes are checked. Both return a DownloadResult with the same facts as the CLI’s JSON result.

Content hashes and entity tags

Every saved file has a content hash, its tag: the SHA-256 of its bytes for a file of up to 4 MiB, and for a larger file the SHA-256 of the SHA-256 hashes of its 4 MiB pieces, in order. A file’s entity tag, as reads and links return it in ETag, is the tag and the length together:

"<tag>-<length>"

Two files with the same entity tag hold the same bytes, in every version that holds them.

​Upload files

​Continue an interrupted upload

​Download files

​Continue an interrupted download

​Share a download link

​Download from code

​Content hashes and entity tags