Baidupcsgo Upload

1. Install the binary

Use the bundled installer script:

bash scripts/install_baidupcsgo.sh

The script prints the absolute binary path on success. Default install directory: ./baidupcsgo

Default release: v4.0.1

2. Log in with browser cookies

Prefer cookie login over username/password.

1. Ask the user to export the Cookie header from an already logged-in pan.baidu.com browser session.
2. Put the cookie string into a local file such as baidu_net_cookie.txt.
3. Run:

COOKIE=$(tr -d '\n' < baidu_net_cookie.txt)
/absolute/path/to/BaiduPCS-Go login --cookies="$COOKIE"

4. Verify:

/absolute/path/to/BaiduPCS-Go quota
/absolute/path/to/BaiduPCS-Go pwd
/absolute/path/to/BaiduPCS-Go ls /

Successful login should show the real Netdisk root and the account quota.

3. Create the remote directory tree

Before uploading, explicitly create the remote path if it may not exist:

/absolute/path/to/BaiduPCS-Go mkdir /output
/absolute/path/to/BaiduPCS-Go mkdir /output/stage1_2b
/absolute/path/to/BaiduPCS-Go mkdir /output/stage1_2b/my-run

Ignore 31061 文件已存在.

4. Upload a whole directory

Use the bundled uploader script when the goal is “local directory -> same-named remote directory”.

Example:

bash scripts/upload_dir.sh \
  /abs/local/output/stage1_2b/coco_bbox_max60-coco80-desc_first-1024-lvis_proxy-merged \
  /output/stage1_2b/coco_bbox_max60-coco80-desc_first-1024-lvis_proxy-merged \
  /abs/path/to/BaiduPCS-Go

The script:

• ensures the remote parent directories exist
• uploads the directory to dirname(REMOTE_DIR) so the original folder name is preserved
• defaults to conservative settings with --norapid -p 1 -l 1 --retry 8
• can be tuned with environment variables instead of editing the script

Example with parallel uploads across files while keeping --norapid:

BAIDUPCS_UPLOAD_PARALLEL_FILES=4 \
BAIDUPCS_UPLOAD_FILE_THREADS=1 \
bash scripts/upload_dir.sh /abs/local/dir /output/stage1_2b/my-run /abs/path/to/BaiduPCS-Go

Example with more aggressive per-file parallelism:

BAIDUPCS_UPLOAD_NO_RAPID=0 \
BAIDUPCS_UPLOAD_FILE_THREADS=4 \
BAIDUPCS_UPLOAD_PARALLEL_FILES=2 \
bash scripts/upload_dir.sh /abs/local/dir /output/stage1_2b/my-run /abs/path/to/BaiduPCS-Go

Use the aggressive mode only when higher throughput matters more than the safer --norapid path.

5. Download a whole directory

Use the bundled downloader script when the goal is “remote directory -> local parent directory while preserving the remote folder name”.

Example:

bash scripts/download_dir.sh \
  /output/stage1_2b/coco_bbox_max60-coco80-desc_first-1024-lvis_proxy-merged \
  /abs/local/output_cache \
  /abs/path/to/BaiduPCS-Go

The script:

• sets BaiduPCS-Go's save directory to the provided local parent directory
• downloads with --fullpath so the remote folder structure is preserved
• defaults to parallel download settings --mode locate -p 4 -l 2 --retry 8 --ow --mtime
• may still create an account-prefixed staging directory such as 1592545883_Pien1722/output/... under the chosen local parent before the final files are merged into the repo tree

This account prefix is a BaiduPCS-Go download behavior, not a transfer failure. Treat it as the staging area for the current login session.

Example with higher download parallelism:

BAIDUPCS_DOWNLOAD_THREADS=8 \
BAIDUPCS_DOWNLOAD_PARALLEL_FILES=4 \
bash scripts/download_dir.sh /output/stage1_2b/my-run /abs/local/output_cache /abs/path/to/BaiduPCS-Go

6. Run inside tmux for large transfers

For multi-GB uploads, prefer:

tmux new -s baidupcs_upload
bash scripts/upload_dir.sh /abs/local/dir /remote/dir /abs/path/to/BaiduPCS-Go

For downloads, the same pattern applies:

tmux new -s baidupcs_download
bash scripts/download_dir.sh /remote/dir /abs/local/parent /abs/path/to/BaiduPCS-Go

Treat tmux as the default launch mode for any long-running transfer. If a command may take more than a few seconds or involve a multi-file sync, start it in a dedicated session rather than the foreground.

Detach with Ctrl-b d.

Reattach with:

tmux attach -t baidupcs_upload

7. Verify completion

Check the target directory:

/absolute/path/to/BaiduPCS-Go ls /output/stage1_2b/my-run

For a large upload, validate the highest-risk artifacts first:

• model shard files such as model-00001-of-00002.safetensors
• tokenizer/config files needed for loading
• index files such as model.safetensors.index.json

For a large download, validate:

• the expected top-level folder exists under the local parent directory
• shard counts and file sizes match ls output from the remote side
• loader-critical files such as tokenizer/config/index files are present
• if an account-prefixed staging tree was used, confirm the contents were merged into the intended repo path under output/

8. Failure handling

If login succeeds but uploads fail:

• rerun with --norapid
• keep -p 1 -l 1 before trying higher concurrency
• if you need more throughput, raise BAIDUPCS_UPLOAD_PARALLEL_FILES before BAIDUPCS_UPLOAD_FILE_THREADS
• verify the remote path is under the real Netdisk root, not /apps/bypy
• verify quota with quota

If downloads fail:

• try BAIDUPCS_DOWNLOAD_MODE=pcs when locate hits authorization issues
• reduce BAIDUPCS_DOWNLOAD_THREADS and BAIDUPCS_DOWNLOAD_PARALLEL_FILES if the transfer becomes unstable
• keep --mtime enabled unless you specifically want fresh local timestamps

If bypy already uploaded files somewhere else:

• remember that BaiduPCS-Go will not show files stored only under bypy's app root
• recreate the intended target path under the real root and upload again

Scripts

• scripts/install_baidupcsgo.sh: download and unpack the tested Linux amd64 release
• scripts/upload_dir.sh: create the remote directory chain and upload a local directory with stable defaults plus env-based parallel tuning
• scripts/download_dir.sh: download a remote file or directory into a chosen local parent directory with env-based parallel tuning