Various toolchains have been set up to facilitate daily chores. However, setting up and maintaining the environment becomes troublesome. Here, some common commands and methods are summarized.

Ubuntu

  • Change the apt-get environment to aliyun. First back up the original list.
sudo cp /etc/apt/sources.list /etc/apt/sources.list.bak

Then, edit /etc/apt/sources.list. The content of the file can be found in the aliyun homepage. Finally, run

sudo apt-get update
  • Grant user permission with sudo chown
    It is used to temporarily grant a super user permission.
sudo chrown -R $USER [FOLDER]

  • Monitoring system resources, a better way: bashtop

  • GPU monitoring: nvidia-smi

  • Add to user $PATH

sudo nano ~/.bashrc

Edit, and add the following line to ~/.bashrc

export PATH=$PATH:/some/directory

Then,

source ~/.bashrc
echo $PATH

Then the directory added should be there in the $PATH variable.

WSL

Always use WSL2

  • Check current version
wsl.exe -l -v
lsb_release -a
  • Show remote (WSL) directory in windows explorer
explorer.exe .

Thanks to Hongjia Liu from the Department of Radiotherapy.

  • Basic data science environment
sudo docker run --name [DOCKER_NAME] -p 10000:8888 -e JUPYTER_ENABLE_LAB=yes -v [LOCAL_DIR:~/docker]:/home/jovyan/work jupyter/datascience-notebook:latest
  • GPU-enabled, advanced data science environment
    An environment integrating zsh, tensor-flow, pytorch, and etc.
sudo docker run -d --name <CONTAINER_NAME> -p <LOCAL_PORT>:8080 --gpus all -v "<LOCAL_DIR>:/workspace" --shm-size 10240m --env AUTHENTICATE_VIA_JUPYTER="<TOKEN>" mltooling/ml-workspace-gpu:0.13.2

<TOKEN> is a paraphrase to assess the container. The CUDA version of the host server must be CUDA-11.2. The host server shall not be an LXC virtual machine.

Jupyter Notebook

  • To set breakpoint
from IPython.core.debugger import set_trace

set_trace()
  • A workaround to skip a certain cell when running the notebook
%%script skip-cell
  • To use a conda environment as a kernel
conda info --envs
conda activate myenv
python -m ipykernel install --user --name myenv --display-name "Python (myenv)"

# To uninstall
jupyter kernelspec list
jupyter kernelspec uninstall myenv

Python

  • Useful pip mirrors
# 清华源
pip config set global.index-url https://pypi.tuna.tsinghua.edu.cn/simple
# 阿里源
pip config set global.index-url https://mirrors.aliyun.com/pypi/simple/
# 腾讯源
pip config set global.index-url http://mirrors.cloud.tencent.com/pypi/simple
# 豆瓣源
pip config set global.index-url http://pypi.douban.com/simple/

Git

  • Proxy
git config --global https.proxy http://127.0.0.1:1080
git config --global https.proxy https://127.0.0.1:1080
git config --global --unset http.proxy
git config --global --unset https.proxy
  • SSH key

In git bash

ls -al ~/.ssh
ssh-keygen -t ed25519 -C "[email protected]"
ssh-agent
eval "$(ssh-agent -s)"
ssh-add ~/.ssh/id_ed25519
ssh -T [email protected]
  • Change remote URI

Github recently ask clients to use ssh everywhere. It is worthwhile to switch from HTTPS to SSH.

git remote -v
# View existing remote

git remote set-url origin git://new_uri.git
# Change

git remote -v
# Validate
  • Repair lost .git folder (as in this post)
# Create a new .git
git init
# Create objects for local files
git checkout -b temp
git add .
git commit -m 'importing local files'
# Configure your remote
git remote add origin [email protected]:xxx/xxx.git
# Now, have it grab the rest of the objects from the remote
git fetch --all
# And switch back to it:
git checkout remotes/origin/main
git checkout -b main
# And, if you didn't have any un-pushed changes:
# delete the local branch
git branch -D temp

R

  • To update R on Windows
install.packages("installr") # if not previously installed
library(installr)
updateR()